Introduction 

In reporting upon Programmed Instruction (referred to subsequently as P.I.), 
we make no claim to represent ‘official’ views from any particular Service 
Department or establishment. We have tried to take account of all docu- 
mentary evidence available to us about Service work in P.I. Nevertheless, what 
follows is drawn principally from our own collective experience, and from our 
general acquaintance with relevant research elsewhere. We have been sparing 
with references throughout the text, but have included a short bibliography at 
the end of Appendix 1. 

As applied psychologists, we are disposed to interpret “research into 
Service training” as “research into the techniques of training by which Service 
objectives are realised”. Our present report is concerned with P.I. as one of 
these techniques of training. Quite early in our discussions, however, we found 
ourselves severely handicapped by the absence of any agreed taxonomy of 
Service tasks or skills. Nor had we any ready-made classification of human 
skills which could immediately be applied in this context. 

In an attempt to overcome this lack we have evolved a provisional two- 
way classification of Training Areas and Instructional Methods. As we hope to 
continue using this model as a guide to our future enquiries, a copy of it has 
been included at page 47. The classification lists different categories of 
military training areas in the left-hand margin, and instructional methods along 
the top margin. The training areas form a comprehensive list of the major ‘fields 
of military training which can be distinguished on an empirical basis. (The list 
does not yet form a taxonomy of actual training tasks, although the develop- 
ment of such a taxonomy is most desirable.) Instructional methods have been 
divided into three groups, based on student-centred, instructor-centred, and 
equipment-centred methods. We have not shown completed entries in the 
various ‘cells’ of the classification; though we envisage using them to indicate 
which instructional methods are in fact, or potentially, in use for different 
training areas. 

Programmed instruction is regarded as a subdivision of student-centred 
methods of instruction. The relevance of P.I. for each of the military training 
areas listed is discussed later. The use of the other training methods will be 
discussed in later reports; and it should eventually be possible to complete the 
cells of the classification table to show which training methods may be appro- 
priate for any military training area. 



The Meaning of Programmed Instruction 

The term is used here to connote a general approach to teaching and training 
which is also identified with such labels as Programmed Learning, Teaching 
Machines, Self-Instruction, and Automated Instruction. These last three terms 
are not, however, synonymous with each other or with P.I. It is by no means 
necessary to present programmes by automatic or even mechanised devices; nor 
are all forms of self-instruction programmed, in the sense defined below. 
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Instruction, or training, is ‘programmed’ when the subject-matter or con- 
tent of the course displays the following defining characteristics: 

(a) There is a clear-cut statement of objectives, i.e. of ‘terminal 
performance’. 

(b) The material to be learned has been itemised and is presented serially in 
identifiable steps, i.e. frames. 

(c) The actual sequence of frames which any student encounters is con- 
trolled according to rules derived from the particular programming technique 
employed. 

(d) Frequent and unambiguous responses from every student are required 
throughout the whole sequence. 

(e) Feedback of information about the correctness or otherwise of responses 
is given to the student before the next frame is presented. 

(f) A ‘response comparator’ is implicit in (e) above to judge between actual 
and desired (i.e. ‘correct’) responses. This function can be performed by the 
student or by a device. 

These are minimal requirements only. Nevertheless they are rigorous enough 
to allow the unequivocal classification of any educational or training course as 
being an example of P.I. or not. It may also be pointed out that, in principle, 
P.I. should be applicable to any training objective which can be specified. 

Many other characteristics have, however, been claimed at one time or 
another as vital to the concept of a programme. Some of these we acknowledge 
to be highly desirable, but not among the defining characteristics. The specifica- 
tion of objectives in behavioural terms* is probably a prerequisite of the best 
programmes, but may not be entirely practicable on every occasion. A full- 
scale internal and external evaluation , following approved procedures, is another 
example. 

There are several more features concerning which it is now quite clear that 
considerable variation is not only permissible, but necessary; contrary though 
this is to earlier, ‘classical’, expressions of programming requirements. Among 
these features we may single out: 

(a) The theory of learning from which the type of sequencing is deduced. 
For example, this may prescribe the same sequence for all students (linear); 
or a sequence which varies for different students according to their differential 
responses to an immediately-preceding frame (intrinsic branching); or it may 
qall for completely flexible sequencing, dependent upon an accumulated his- 
tory of past responses (adaptive). 

(b) The device by means of which a programme is presented, and through 
which the actual control of sequences is effected. This may be automatic, 
mechanical, or entirely non-mechanical, in operation. Its displays may be 
visual or aural; or even tactual and kinaesthetic if training objectives imply 
the need. 

* E.g. “After training, the student will be able to measure electrical resistance, voltage, 
current, and valve characteristics, within the (specified) following classes of electronic 
equipment, using the (specified) following measuring instruments . . . .” rather than 
“A trained man must be able to use test equipments efficiently, as a basis for carrying 
out standard maintenance -schedule routines”. 
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(c) The extent to which a student’s rate of progress through a programme is 
under his own control. In ithe early days of P.I. it became almost an ‘article 
of faith’ that self -pacing was an indispensable feature. The evidence for this 
has never been extensive nor particularly convincing; and nowadays the 
insistence on unrestricted self-pacing is slackening. Whilst the customary 
‘lock-step’ pacing of a conventional classroom group is just as much to be 
deprecated as ever, the possibility is beginning to be recognised that some 
forms of group-controlled pacing, and ‘adaptive’ pacing for individuals, are 
potentially advantageous. 

(d) Response mode, step (and frame) size: these aspects of P.I. were at one 
time the centres of lively controversy. It now seems unwarranted to take up 
a dogmatic position with respect to either of these issues. Decisions about 
whether or not to use constructed, discriminatory, or multiple-choice, re- 
sponses cannot yet be made on theoretical grounds; they should be influenced 
by the total situation in which a programme is being prepared, and par- 
ticularly by the nature of the terminal behaviour to be acquired. Similar 
attention to all the circumstances should precede decisions about how much 
information appears on a frame, and what increments are acceptable between 
frames. And in either case, decisions should be open to revision in the light 
of evaluative data. 

A Summary of PX Studies in the Services 

The Service departments have been pioneers in the United Kingdom with 
respect both to programme writing and to studies of presentation devices and 
productive ways of using P.I. Most of this work is documented; some of it has 
been published in the appropriate journals. However, we regarded it as desir- 
able to bring together as much of the reported data as seemed relevant to our 
present brief. Since some readers may be familiar with some of this work, we 
have included the summary as Appendix 1. 



Trends in P.I. Research and Development 

In this section we wish to draw attention to certain trends which are sufficiently 
pronounced for there to be no doubt of their relevance to Service progress in 
utilising P.I. Not surprisingly, the United States is where the trends emerge 
most clearly. However, what follows is also represented strongly in current 
interest and discussion in the United Kingdom, though it is not necessarily 
indicative of where most effort is now being expended. 

The Concept of a Technology of Training 

Refinement of technique in programme writing and evaluation has been more 
conspicuous than concern with the economics and administration of P.I. Never- 
theless, attention has been directed increasingly to some of the concomitant 
factors which condition the effectiveness of P.I. In those cases where the whole 
field of design and practical operation of a training course has been studied 
( ... not only programmed, but ‘conventional’ courses too), the efficiency of 
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training has clearly been shown to depend on many factors. The quality of 
instruction is only one of these factors. The relevance and reliability of pro- 
ficiency measures, during and after training, is another. 

Some of these factors can be identified as stages in the actual development 
of a well-designed training course. One expression of these succeeding stages 
might be: 

(a) task and behavioural analysis of the job to be learned, 

(b) declaration of training objectives, in behavioural terms, and construc- 
tion of a criterion test of acceptable performance, 

(c) determination of the knowledge and abilities available within the 
population of potential trainees, 

(d) specification of the training course, including prescription of content 
and practice of instructional methods such as P.I., 

(e) assessment of training effectiveness, using objective proficiency meas- 
ures and cost criteria. 

There is a wide measure of acknowledgment in the United States that 
maximal efficiency in training is unattainable unless all these stages have been 
undertaken. What many psychologists refer to as a ‘technology of training’ 
consists of the various techniques and professional methods which have been 
developed to accommodate these stages in the design and construction of a 
training course. Not all of these techniques are new, nor are they all highly 
developed. Perhaps the most critical in terms of ultimate success in writing 
programmes (or preparing any method of instruction) are those which lead 
to the declaration of training objectives in behavioural terms, i.e. to ‘terminal 
behaviour’. The terms ‘behavioural analysis’ and ‘task analysis’ are commonly 
used of the various procedures which are used to define terminal behaviour. 
By ‘task analysis’ is meant a rigorous psychological analysis of what has to be 
learned, taking as the starting point an objective study of the knowledge and 
skill required in the operational situation itself. ‘Behavioural analysis’ is some- 
times used to cover not only the breakdown in behavioural terms of the tasks 
to be learned, but also of the intermediate stages of knowledge and perform- 
ance through which a trainee must pass in order to attain the terminal 
behaviour itself. 

The concept of a training technology is particularly relevant to our pres- 
ent topic, since the factors summarised above can almost be regarded as a 
paradigm for P.I. itself. It is indeed largely due to the academic and practical 
interest aroused by P.I. that the notion of a general technology of training, 
with its much wider implications, is beginning to take root. As this happens, 
we may expect to hear less expression of the misguided view that P.I. oilers 
a straightforward solution to all our training problems; or, conversely, that it 
threatens the professional teacher and trainer with a form of automation at its 
worst. A glance at page 47 should leave no doubt that there are many other 
components of training than P.I., at least as we know it today. 

Extensions of P.I. to Various Training Areas 

During the first decade of P.I., programmes have generally been confined 
within two training areas which we have labelled acquisition of information 
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and intellectual skills (see page 47). Academic and commercial interests have 
together predisposed programmers to concentrate upon educational subject- 
matters, ‘background’ factual information relevant to particular vocational 
fields, and intellectual skills such as mathematics and electronic fault-finding. 
Most of the U.K. Service applications so far have also been directed towards 
academic, theoretical, and informational, aspects of technical training. 

There are obvious reasons why the P.I. techniques as developed at present 
should continue to be applied principally in these fields. If there is one undis- 
puted finding from our experience to date, it is that the effort and time 
required to prepare a good programme is so great as to deter its being 
expended (at least within the Services) on training objectives which are liable 
to radical and frequent change. Likewise it is most profitable to direct pro- 
gramming effort towards courses with a large and steady through-put of 
students, where there is promise of reducing overheads in terms of failure- 
rates, training-time, and instructional staff. 

The potential for P.I. which remains in these two training areas is so 
great within each Service as to exhaust more than all the programming effort 
which can reasonably be anticipated. We are not in a position to judge just 
how far it is practicable to agree upon treatments of subject-matters which are 
ostensibly common to all three Services. However, we are convinced that 
serious attempts should be made to co-ordinate the programming work which 
each Service undertakes in the aforementioned aspects of basic and advanced 
technical and educational instruction. 

Without wishing to retract our statement that P.I. is by no means a 
panacea for all training problems, we nevertheless regard it as important that 
the Services should not fail to take advantage of other developments in P.I. 
technique, outside the two training areas just mentioned. As we have pointed 
out in the Appendix 1 (para. 28), the basic principles behind P.I. are not 
exclusive to the learning of verbal knowledge and intellectual skills. They are 
valid for manual and perceptual tasks, too; perhaps for some non-cognitive 
objectives also. However, the majority of skills listed on page 47 are highly 
specific in content. Certainly this is so of most procedural, perceptual, and 
control, operations; and it is probably characteristic of many managerial 
skills and group activities. If only for this reason, applications of P.I. within 
these categories are subject to more stringent demands of task and behavioural 
analysis. Moreover, the programmed courses, and presentation devices (which 
are likely to entail close simulation), are unlikely to be generalisable over a 
wide area of common Service use. It was probably a failure to take account 
at least of the former limitation that led to disappointing results from early 
American ventures in the programming of perceptual skills. 

Nevertheless the majority of Service training is actually (or should be) 
concerned with the specific and practical aspects of operational activities, over 
and above the underlying theoretical and informational aspects. It is there- 
fore incumbent upon us to investigate the extent to which the demonstrable 
benefits of P.I. can be extended to these other aspects. Hopes of realising this 
possibility are a potent factor in the recent upsurge of industrial interest in 
programming. So also have the Services begun to take action in this direction. 
Two recent examples may be cited. 
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The first is referred to in Appendix 1 (para. 15). It was soon realised in 
HMS COLLINGWOOD that the maximum benefit from their electronic 
maintenance courses could not be extracted whilst P.I. was restricted only to 
the ‘theoretical’ content, whilst practical and laboratory work was continued 
under conventional procedures. Rates of progress throughout a technical 
course remain contingent, under these circumstances, upon the organisation 
of group laboratory sessions; supervisory overheads are also higher than may 
be possible if the entire technical training is programmed. Though it is early 
yet to pronounce on the matter, there seems every likelihood (judging from 
experience in HMS COLLINGWOOD) that significant improvements will 
follow, in productivity as well as in the attainment of technical skill, from the 
programming of laboratory experiments and of instruction in the use of test 
instruments. 

A second innovation is also noted in the Appendix (paras. 28 to 31). It is 
a project which has the object of developing distinctive P.I. courses for per- 
ceptual tasks such as occur in the classification of underwater targets. A 
feature of this adaptation is that P.I. techniques of cueing, feedback, and 
sequencing have to be associated with an appropriate simulation of critical 
task and equipment characteristics. This means that fairly sophisticated 
teaching machines, perhaps with auditory as well as visual displays, are likely 
to be indispensable. We should expect this requirement for relatively complex 
instrumentation to be characteristic of successful P.I. for control skills as well 
as for perceptual ones. 

Another training area in which successful application has been demon- 
strated is that of procedural skills. An early and notable American example of 
programming a routine sequential task, where the elements are discrete, is 
afforded by the Hughes Aircraft Company’s electronic assembly line. Here 
the operatives’ work is directed by means of individual Videosonic teaching 
machines*. In many situations like this, it may not be necessary to learn the 
sequence, provided the elements themselves can be performed precisely and 
carried out under directions. This may be true of some standard maintenance 
schedules, switching-on drills, clerical routines, stores inspections, and many 
other Service procedures. In any event, whatever the extent to which the 
sequences, as well as individual elements, have to be learned and remem- 
bered, all these tasks lend themselves very well indeed to more effective and 
economic accomplishment through the medium of programmed materials. Nor 
is it necessary to envisage expensive or complicated teaching machines to 
accommodate the majority of such programmes. 

We have already noted the extension of P.I., at present still under 
development, to perceptual skills in the sonar operating field. Training for 
perceptual tasks covers a wide range of objectives, from aircraft recognition 
and photographic interpretation, to visual and aural search and classification 
in a multiplicity of surveillance, weapons, and communication, systems. The 
basic ideas of P.I. and the specific cueing techniques which have become 
associated with discrimination training generally, are directly relevant to the 



* A recent British product, which appears to be similar in principle, is the Smith’s 
Audio Diagnostic Instructional Equipment (SADIE), proposed for use in troop transport 
aircraft as an aid to maintainers. We are not in a position to judge whether or not this 
device successfully embodies all the requirements of a procedural P.I. system. 
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instructional problems of these critical operating skills; and we should advo- 
cate that every opportunity should be taken when perceptual training equip- 
ments are being developed, to incorporate these improvements. 

The training area which we have called control skills is also a very wide 
one. Within it we have distinguished craft, operating and coding skills. The 
first of these is an industrial commitment, too; and the recent urge to improve 
industrial training facilities for apprentices has produced the first attempts to 
apply P.I. for this purpose. The informational aspects of craft training (includ- 
ing workshop technology data) are naturally amenable to instruction by pro- 
grammed, self-instructional, methods. It also seems, from the pilot demon- 
strations produced by a firm called Learning Systems Ltd., that machine-tool 
(as well as hand-tool) operations can be taught with the aid of programmed 
texts or teaching machines. Since one of the aims of this new application is to 
ease the supervisory load during workshop training, we hope the Services, for 
whom the teaching of craft skills represents so large an investment, will them- 
selves take part in investigating and developing it. An exploratory study in 
this field has in fact already been started in association with the Navy’s 
apprentice training establishment. 

Operating and coding skills, which are exemplified respectively by con- 
tinuous tracking tasks and by keyboard tasks are not likely to offer scope for 
the simpler techniques of P.I. Both these areas, however, continue to be the 
subject of research designed to produce appropriate ‘adaptive’ programming 
devices for training. (Adaptive P.I. is introduced in the following section.) 
The Army has already sponsored a university research into adaptive training 
of 2-dimensional tracking. One of the difficulties here lies in the prior task 
analysis; the structure of a skill must be specified before the design of a suit- 
able ‘adaptive’ trainer can proceed. It is not yet determined whether there is 
enough common ground among tracking, or keyboard, skills to make it worth- 
while to employ ‘generalised’ trainers for the personnel who must master 
various individual tasks. In any event, one of the characteristics of an adequate 
instructional system for any of these skills is almost certain to be an ‘adaptive’ 
capability. 

Most of the effort in industrial as well as military training, so far as P.I. 
is concerned, will continue to be taken up in the five training areas we have so 
far surveyed. Among the remaining six areas in the list on page 47, a serious 
interest in P.I. has so far only been evident with respect to managerial skills. 
This is principally due to industrial prompting. However, the Services have 
been equally involved in management training and will presumably expand 
even further in carrying out this function as the load of technical expertise 
increases. So far as we are aware, the only commercial programme on 
management which has been used in a Service context, in the Army, is the 
U.S. Industries “Effective Executive Practices”; this programme seems to have 
demonstrated some usefulness in an industrial context. However, we cannot 
from present evidence hazard more than a guess that P.I. could make a worth- 
while contribution to Service management training. Programmes on the 
‘tools’ of personnel and technical management (examples could be PERT and 
MILITARY LAW) would no doubt be relevant and helpful. Whether a 
contribution based on P.I. could extend beyond the informational basis of 
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leadership, decision-making, or staff activity, is a matter for conjecture until 
some preliminary research has been carried out into the nature of the require- 
ments. 

The situation appears to us substantially the same with respect to the 
remaining five training areas, i.e. physical fitness, character-building, identi- 
fication with the group, system activities, and operational formations. Non- 
cognitive components are dominant among three of these, and they are also 
critical to system activities. Surprisingly, perhaps, the attainment of physical 
fitness may well be the area in which programming principles would most 
probably repay application. One of the newest avenues to athletic skill (‘circuit 
training’), though constructed in complete independence of P.I. influences, in 
fact incorporates several key features of a programmed course. It may well 
be that its success is attributable to these (knowledge of performance, regular 
experience of success, and an ‘itemised’ sequence of activities); and that 
further improvements are possible from a more deliberate integration of the 
two approaches. 

System activities are clearly built on foundations of individual skills, 
which are themselves likely to be amenable to P.I. but the distinctively ‘group’ 
aspects of system activity present training problems for which accustomed 
forms of P.I. will probably not be very helpful. At present, although there have 
been no Service investigations of the medium, it seems that ‘adaptive’ group 
teaching machines (see the following section) offer the most likely outlet for 
applying P.I. techniques. 

There is so little which we can say with any confidence about the other 
areas, with respect to P.I., that we think it wise to reserve further comment. 
What does seem to us to be notably lacking in these areas is a reliable corpus 
of systematic research information, based upon U.K. experience, from which 
to begin an assessment of whether or not P.I. has any distinctive contribution 
to offer. 



‘Adaptive’ and Computer-based Instructional Systems 

What we have discussed represents a selective view of current ideas and 
developing practice within the ‘orthodox’, or generally-accepted, framework 
of P.I. (By ‘orthodox’, we mean no more than that the principles are based 
upon one or another paradigm which is conspicuous in psychological learning 
theory, and that the instructional sequences followed are either pre-determined 
completely or are subject to variation within narrow limits set by an immedi- 
ately-preceding response only.) 

Very early in the ‘history’ of P.I., however, the cybernetic view of a 
learning system, in which the accumulated record of past responses is at all 
times influential in guiding future states of the system, began to attract notice. 
The pioneer in this respect has been Gordon Pask, of System Research Ltd., 
in the United Kingdom. Programmes had to be ‘adaptive’, i.e. flexible in the 
sequencing they permitted, if they were to, meet the criteria of a cybernetic 
model. In this section, therefore, we shall comment on some relevant features 
of this approach to P.I., which also affords a logical introduction to the ideas 
which lie behind computer-aided instructional systems. 
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' Adaptive ' Teaching Machines. Devices which are ‘adaptive’ in the sense 
used by Gordon Pask and his associates may be briefly characterised as 
follows: 

(a) The difficulty level of the lesson is continually adjusted to suit a wide 
range of individual ability — i.e. to maximise the rate of improvement. 
Typically, cues are introduced or withdrawn, and the pace is varied. 

(b) In order to achieve (a), responses must be classified into types and 
information about each type separately stored. The interpretation of this 
information determines the course of the lesson for a particular individual, 
i.e. the device bases the lesson on what it can learn about its pupil. 

(c) Interpretation of response information may consist of built-in decision 
rules or, it is claimed, the device itself may select and modify decision rules 
on the basis of their past effectiveness, i.e. it may base the lessons on what it 
has learned about a population of pupils. 

Apart from the superior capacity to cope with individual differences, 
which is implicit in the foregoing characteristics, adaptive teaching machines 
are said to have a superior motivational property. Pask speaks of “maintaining 
the requisite variety”. Lewis is more explicitly psychological: “Now it so 
happens that humans have an irresistible tendency to withdraw (through bore- 
dom) from tasks that are too easy and to withdraw (through frustration) from 
tasks that are too difficult, . . . machines must therefore determine ... a level 
of difficulty which the student regards as being neither too easy nor too 
difficult”. (Lewis, 1963a.) 



This position on motivation is a very plausible one, but it should be 
recognised as a credo. One might argue that the appeal of “a level of difficulty 
which the student regards as being neither too easy nor too difficult” may lie 
in a tendency of current training methods to deprive the trainee of this state 
of affairs; and that a motivational effect produced by deprivation may be 
subject to satiation. (It has been argued that a high level of correct responses 
in a Skinnerian programme is therefore only apparently and temporarily 
reinforcing.) The point to note here is that a priori arguments about motiva- 
tion are best distrusted until shown to be demonstrably true in extended 
empirical studies. 



It has nevertheless been claimed that devices exhibiting adaptive charac- 
teristics have been realised for the following skills: 



(a) Keyboard Skills: SAKI is the well-known example. 

(b) Radar Detection: of a wide range of faint signals, presented at 

irregular intervals, against varying noise back- 
grounds. 



(c) Radar Tracking: of targets of varying ‘elusiveness’. 

(d) Fault Finding: i.e. correction of incorrect diagnoses and search 

strategies. 



(e) 'General 

Comprehension’ 



of verbal material, by means of multiple-choice 
questions. 
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However, whilst these devices have been constructed and do, in fact, exhibit 
‘adaptive’ characteristics, evaluation studies establishing their effectiveness or 
efficiency are lacking. The middle three, particularly, are not very promising 
for Service purposes, since they are based upon assumptions about the under- 
lying generality of perceptual and control skills which do not stand up to the 
test of practical experience. 

Adaptive teaching machines should not be thought of as applicable solely 
or mainly to the learning of skills, rather than of knowledge; the feasibility of 
their employment in the same areas as, for example, Skinnerian or Crowderian 
devices, is claimed by their proponents. It is, after all, legitimate to claim that 
all the criteria of a ‘programme’ (cf Introduction) are complied with in adap- 
tive instructional sequences. 

A point which should not be overlooked, however, is that the response 
comparator function is built into a device such as SAKI; and much of the 
power of adaptive machines may lie in this special feature. In the acquisition 
of general knowledge the response comparator function could probably not be 
handled by the device itself but would have to be handled in some other way, 
e.g. by either (i) a programme writer, who would construct multiple-choice 
questions; or (ii) the student, who would evaluate his own responses; or (iii) 
a combination of programme writer and student, i.e. the latter, having re- 
sponded, would be presented with a set of possible responses and be required 
to indicate which of these he had in fact made. (Inevitably not all responses 
would be foreseeable and a ‘catch-all’ category would be necessary. Neverthe- 
less, whenever the responses in this category become numerous, they can be 
analysed to establish an improved set of response categories.) 

More recently Task and his associates have studied possible functions of 
adaptive devices in group learning. In particular, they have done work on an 
adaptive device which mediates the instruction of members of a group by one 
another (Lewis, 1963b); and have reported the effects of having a number of 
subjects, who are required to function as a group, communicate with each 
other via an adaptive device (Lewis and Pask, 1964). In the latter case 
the adaptive controller enables the members of the group to develop 
a common language in dealing with a given problem; improves mastery of 
the language by having ‘transmitters’ play the role of ‘receivers’; and tends to 
produce a language which is better able to handle changes of strategy, which 
enables group functioning to recover more quickly following disturbance by 
‘misinformation’, and which is less variable in communication time. 

Research studies of the value of adaptive teaching machines in Service 
training are probably overdue. These devices might, in the first instance, prove 
more powerful in those training areas where a response comparator function 
could be built into the devices, i.e. in craft, control, perceptual and possibly 
some intellectual skills (see page 47). Recent work by Pask and his associates 
may ultimately have relevance for management skill and system activity. 

Computer-Assisted Instructional Systems. (C.A.l.S.) In an experiment by 
Smallwood (Smallwood, 1962) subjects learned “a miniature finite Geometry” 
via an electric typewriter, a microfilm reader, and the Massachusetts Institute 
of Technology computer (I.B.M. 709). The experiment established that the 
programme achieved the decision structure intended by the experimenter, i.e. 
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It based its lessons both on the past performance of each individual while he 
was under instruction and on the performance of the several individuals who 
were instructed. This experiment is cited because it : 

(a) exemplifies the functioning of a computer-based system in the manner 
designated as ‘adaptive’, and 

(b) demonstrates that adaptive characteristics can be realised in a device 
which is controlling ideational learning, i,e. intellectual skills and 
acquisition of knowledge. 

A considerable amount of work in America with teaching systems based 
on digital computers has been reported. Such systems have usually been used 
to teach ‘intellectual’ material, and their feasibility with this sort of material 
is fairly well established. Among the best known examples of advanced 
C.A.I.S. are the System Development Corporation’s CLASS and the Univer- 
sity of Illinois’ PLATO and SOCRATES systems. There are at least half-a- 
dozen other experimental but sophisticated computer-based systems in the 
U.S.; though there are none yet in Britain. 

It is possible that, for the higher cognitive functions at least, such 
computer-based systems may make the realisation of adaptive characteristics 
more feasible or more economic, especially if the time-sharing problem can 
be solved. 

Research Potential of C.A.I.S. A most intriguing feature of computer-based 
teaching systems is that typically the subject matter is stored separately from 
the rules of instruction, e.g. on slides, film, or even in programmed texts and 
conventional books. The rules of instruction, usually a computer programme, 
can, for example, be varied whilst the same ‘frames’ are employed; or they can 
be used with different sets of ‘frames’, covering different subject-matters, or 
different treatments of the same subject-matter. 



It seems likely that the construction, testing and revision of computer 
programmes embodying an instructional logic may prove more powerful 
in determining effective instructional procedures than current psychological 
theory. Furthermore, it should not be supposed that the acquisition of the skills 
necessary to the production of such computer programmes need present any 
great difficulties to psychologists and other workers in the field of programmed 
instruction. Programming languages, specially developed for laymen, such as 
FORTRAN, COBOL and ALGOL, have in recent years greatly extended the 
usefulness of the digital computer to business, and to scientific research in 
general. Comparable languages will doubtless be developed for the program- 
ming of C.A.I.S. Indeed work has already been done on at least two such 
languages: JOVIAL, which is being developed by the System Development 
Corporation, and DIALGOL, so named because its formal similarity to 
ALGOL is such that it may be regarded as a ‘dialect’ of that language. If 
viable specialised languages are successfully produced for computer-aided 
teaching systems, they may eventually make possible the discovery of effective 
instructional procedures of some generality. 
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The International Business Machines corporation, which is a leading 
participant in the U.S. drive towards expanding research and development in 
C.A.I.S., has recently developed a ‘language’ known as COURSEWRITHR. 
This is a facility which allows a teacher with no special knowledge of com- 
puters to present a course of instruction through a computer and receive 
records of student performance. We suspect that this development may 
promise a breakthrough in programme writing, i.e. in the actual ^preparation 
of programmes which can subsequently be presented by orthodox means. At 
present, programme writing constitutes the most difficult logistic problem in 
the widespread implementation of P.I. Any advance which would alleviate 
this problem should, in our opinion, be investigated as a matter of some 
priority. 

Applicability of C.A.l.S. to Service Training. At the present time, computer- 
aided systems are the subject, as well as a tool, of research into training 
problems. Yet their potentialities for future exploitation in the service of 
instruction itself are already clear. For this reason we are strongly of the 
opinion that the most outstanding, and indeed urgent, need for advanced 
military research in P.I. is to provide the resources for studying C.A.l.S. in 
action. The main justification for this proposal is that a suitable computer- 
based installation would permit applied research directed towards any of the 
following several objectives, each of which is, in our opinion, indicative of 
what could become accepted practice ten years hence : 

(a) Investigation and adaptation of COURSEWRITER as a possible means 
of compiling programmed material for P.I. using existing styles and 
presentation devices. 

(b) Use of the installation to derive empirically the optimal instructional 
‘strategies’ for specified training requirements and syllabi, perhaps 
using selection and other test results as part of the determining data. 

(c) Experimental use as a control system for guiding and routeing students, 
in adaptive sequences, through instructional courses for which the 
sources of information, and training aids, are external to the computer 
itself. These sources may include programmed texts, simple teaching 
machines, as well as ‘conventional’ media like handbooks and illus- 
trations. 

(d) Experimental use as a control system for groups of students, each under- 
going instruction with the aid of individual teaching machines which 
act as the information stores and presentation devices. The system 
might be extended, on a time-sharing basis if necessary, to aid the 
administration of a training system, e.g. by compiling individual student 
records and regulating the progress of the group within acceptable 
limits of variation. 
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Problems of Military Research into Programmed Instruction 

It will be seen from Appendix 1 that research in the Services during the last 
few years has confirmed American experience that P.I. is a reliable method of 
instruction, and' that its claim to reduce training costs is justified. Neverthe- 
less, in spite of the vast amount of research which has been done in the U.S.A. 
no complete theory of instruction is available which can be used as a universal 
guide in designing training courses. For example, much experimental data, 
and the analysis of student behaviour during P.I., have failed to support the 
early contention that operant conditioning was the appropriate paradigm for 
the new medium. The purpose of this section is to outline some outstanding 
problems of research in this field and to indicate areas where U.K. military 
research might be concentrated. 

Ambiguities have also arisen from a variety of inadequacies in research 
design. For example, it is now becoming apparent that the following factors 
may have rendered many experiments inconclusive: 

(a) The use of single-variable experiments in what are essentially multi- 
variate situations. There has been a lack of systematic variation of 
factors, such as initial attainment, which might be relevant to the 
learning situation. 

(b) The lack of control of interactions between variables, e.g., the inter- 
action between method of presentation and amount of instruction. 

(c) The use of comparative experiments where the standards against which 
P.I. has been compared are unspecified, as in the description ‘conven- 
tional teaching’. 

The use of multivariate experiments implies a greater sophistication in the 
analysis of results, which in turn almost necessarily suggests the use of digital 
computers for data processing and statistical analysis. While the use of com- 
puters for statistical analysis is now a commonplace, their role in the simula- 
tion of learning appears to have been neglected. It remains to be seen how 
far field experimentation in the research on P.I. can be replaced by simulation 
techniques. The use of multivariate analysis, and the systematic manipulation 
of variables across task characteristics and student characteristics, may imply 
co-operation between Services in the provision of adequate student samples. 

Nevertheless, in spite of these difficulties a number of experimental results 
are emerging which can give rise to a conceptual framework for education and 
training, even if this is limited to sub-areas of knowledge rather than amount- 
ing to a general theory. For example, the research which has been done on 
the need for overt responding shows that the original claims for P.I. were 
misleading, in the sense that learning will take place even if the response is 
covert; the universal employment of active responding is therefore unnecessary, 
even though it appears that there are certain circumstances where an overt 
response does facilitate learning. When the characteristics of these situations 
have been fully explored, it should be possible to build a satisfactory sub- 
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theory concerning the role of response in learning; though such a theory is 
likely to be far more complex and specific than the Skinnerian model which 
preceded it. 

Thus, the military psychologist finds that there is no fruitful general theory 
on which applications for military training can be usefully based, and that to 
test hypotheses (whether arrived at intuitively or as a consequence of theory) 
demands experiments which are not easy to design and put into practice, 
because of the multivariate nature of the learning situation. In addition, the 
applied nature of his work forces him to “get programmed learning going” 
whilst some of the critical features of the method are still in dispute. Judg- 
ments often have to be made about the value of different types of presentation 
devices and programmes, and about the rules for programme writing, without 
the experimental results to support these decisions. 

Because military training is so expensive, it is understandable that any 
improvement which produces a reduction in cost is likely to be pressed on the 
training authorities, even though it is not perfected and therefore may fall 
short of optimal efficiency. The danger to avoid is that these less-than-optimal 
improvements will be accepted as sufficient for too long, to the detriment of 
future innovations which may have much more to offer. 

When we consider in what areas of P.I. should military research be concen- 
trated, there are two constraints which affect the answer. In the first place, 
military psychologists are working in an applied field, and their work is more 
likely to be supported if it is seen as helping to solve military problems, even 
long-term ones, rather than being concerned with academic research. Secondly, 
since many of the problems of military training are not likely to be the direct 
concern of non-military research workers, it is desirable that military research 
should tend to concentrate on those situations which are more generally found 
m the Services than in civilian training situations. Military tasks cover a range 
of skills which are either absent or unimportant in civilian occupations. It 
would be appropriate, therefore, to allocate priority to studying the program- 
ming needs of skills which are specifically military, and for which there is little 
or no inducement in civil research organisations to develop P.I. (Some of these 
training areas have been noted in the preceding section.) 

With these points in mind, it is possible to frame a list of topics for research 
which might be the province of the military psychologist rather than, say, the 
university research worker. For the reasons already indicated, we have not 
given priority to the study of stimulus-response variables (such as response 
mode) and programming variables (such as step size) which have hitherto 
formed much of the basic research on programmed learning. 

Such a list might include the following problems : 

(a) The development of more systematic and objective procedures for the 
elements of the Technology of Training. Although it is claimed that a tech- 
nology already exists, several components are only developed to a rudimen- 
tary degree. For example, while it is agreed that a task analysis should 
precede the specification of behavioural objectives, the exact method for 
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carrying out a task analysis is not thereby prescribed. Similarly, the trans- 
lation of behavioural objectives into training syllabi is also a matter for 
subjective interpretation. If non-psychologists are to perform these tasks in 
a technical manner in the future, they would seem to need more rigorous 
definition and systematisation of procedure. 

(b) As has already been noted in the Introduction to this report, there is 
no adequate method yet of describing and classifying task characteristics; 
and yet a thorough-going approach to training research requires more than 
a haphazard grouping of learning tasks, based on superficial characteristics. 
The provision of an adequate structure for classifying military tasks would 
assist the solution of military problems in the field of career structures, as 
well as in programmed instruction and training generally. 

(c) Just as the Services have many jobs requiring special skills, so they also 
have unusual training situations. Thus, there is a very large requirement for 
training in small, remote, and scattered, units where it is difficult to use the 
conventional classroom method of instruction. The needs of the isolated 
pupil working in a technical or military situation should be considered as a 
particular topic for research within the Services. To some extent the Services 
already rely upon self-instruction in these circumstances, through the 
medium of correspondence courses. The value of P.I. in this context has still 
to be determined. 

(d) Because the Services are the natural targets for manufacturers of train- 
ing devices, it is likely that there will be a constant need to perform evalu- 
ative studies of new developments in training ‘hardware’, such as teaching 
machines. While these studies may not have a high status in terms of 
generalisable effects, they seem to be necessary if the Services are to spend 
money only on the most efficient devices. In the early days of P.I. it appeared 
to be satisfactory to ask whether, say, a machine was better than a pro- 
grammed book or conventional teaching. It now seems more appropriate 
to ask what are the unique features of machines as compared with other 
media, and in what training situations will these features repay their possible 
extra cost. 

(e) Military organisations have tended to place great emphasis on exami- 
nations to determine technical qualifications and advancement. The end-of- 
course examination, for example, is an integral feature of military training. 
There is reason to suspect that existing methods of performance assessment 
are inadequate for the evaluation of training or insufficiently relevant to 
operational jobs. For example, examiners may assume that students will 
differ widely in scores, even that a number will fail; and the form of the 
examination is determined by such considerations. Because of the need to 
validate programmes, it is desirable to review critically the current methods 
of performance assessment. 

(f) There has been considerable discussion recently of a ‘systems’ approach 
to training, in which all the media of instruction (P.I., human teachers, 
closed-circuit TV, textbooks, etc.) are used in their appropriate roles in an 
integrated training environment. There is very little evidence, however, as 
to what are the most appropriate roles for these various media. Research is 
needed to delimit the parts to be played in a system (and their inter- 
relationships) by P.I. and other methods. 
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We think that the role of the instructor is particularly deserving of further 
study. Questions about it include: Is conventional classroom teaching re- 
placeable entirely by P.I. reinforced by tutorials, discussion groups, and 
more guided practical work? If something more than an invigilator is 
needed when students are engaged on P.I., need he be a qualified instructor? 
Should he have additional teaching aids, or even programmes which exploit 
his presence? 

(g) Experience has already made it clear that self-pacing (commonly 
thought to be an essential characteristic of P.I.) during training courses can 
produce administrative problems of the first magnitude. This is so at least 
of establishments which are at present bound to accommodate training in 
groups. Yet it is not really established that unrestricted self-pacing is vital, 
even assuming it were possible to achieve in an environment where in-group 
pressures and interactions are present. ‘Lock-step’ pacing which is not 
governed by the responses of students is almost certainly detrimental. But 
if the stringency of pacing within a group can be continually adjusted by a 
teacher, or by a central pacing device, on the basis of feedback about 
responses, then it is arguable that self-pacing may present little advantage. 
It is at any rate an empirical question, requiring evidence, to determine 
whether ‘adaptive’ lock-step pacing can be achieved, with fully effective 
learning. 

The topics listed above are not exhaustive of the outstanding military prob- 
lems of applying P.I.; but, so far as the present generation of P.I. techniques 
is concerned, they appear to us the most pressing. They are, moreover, charac- 
teristic of military, rather than civilian, education and training. Finally, since 
the raison d’etre of P.I. is that it will lead to lower-cost, and therefore more 
efficient, training, it would be desirable to establish the actual training costs 
of the present systems. Although global training costs are known as a matter 
of administrative routine, there is little information about the actual costs per 
student hour in specified training courses, and even less information about the 
operational cost of using badly-trained men. Although we do not regard the 
provision of this information as part of the psychologist’s task, we should like 
to underline the relevance to his work of an examination of costs. 
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Conclusions 

We have reviewed what seem to us the main ‘growing points’ in the theory and 
technology of P.I. which have practical implications for military training. The 
appendix which follows provides a survey of research and development pro- 
jects undertaken by Service Departments to investigate the potentialities of this 
new medium. 

It seems to us from the evidence that in some training areas at least, P.I. 
can be the most effective method available to us. The techniques are demon- 
strably suitable for teaching the academic and technical subject-matters which 
are the basis of Service education, and they are equally so for the informational 
aspects (i.e. ‘job-knowledge’) of operational, maintenance, and managerial 
tasks. Not only is P.I. an effective approach, but it is likely to be more 
efficient, in the sense of achieving acceptable results more economically. 

Whether or not an increased effectiveness is actually attained depends prim- 
arily, of course, upon the quality of programmes. The only guarantee of their 
quality is that every stage of programme writing, from task analysis to evalu- 
ation, has been carried out with professional insight and thoroughness. Nor is 
it to be taken for granted that, once introduced, P.I. will be used with maximal 
efficiency. Exploiting the medium fully is partly an administrative matter, and 
depends upon the relative priorities attached to the following economies; all 
of which, but not all at once, have been realised in practice: 

(a) reduced demands on highly-qualified instructors, 

(b) reduction of training time, due to increased rates of learning, 

(c) attainment of higher standards of knowledge and proficiency, 

(d) increased ‘through-put’ of trainees. 

The training areas noted just previously by no means exhaust the potential 
applications. We should expect equal gains from a systematic use of P.I. to 
achieve proficiency at standardised, routine, drills and sequential procedures. 
Clerical and maintenance routines afford wide scope in this area. We may note 
here that programmed devices can just as well be used in job situations where 
there is no absolute need for the activities to be learned. It may sometimes be 
more efficient just to ensure that the required procedures axe carried out in a 
prescribed way, using the simplest practicable device for presenting a pro- 
gramme of instructions and guidance ‘on the job’. 

Another outlet for increased efficiency and effectiveness in learning could be 
(o use whatever programmes are available in the above topics to permit service- 
men to study at their home or operational stations for examinations which they 
currently prepare for in formal training establishments. Personnel whose needs 
are at present catered for by means of correspondence courses could also benefit 
from the substitution of self -administered P.I. An extension of these proposals 
would be to seek out situations where P.I. can be applied ‘on the job’ itself, so 
that controlled training takes place whilst personnel are performing their 
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operational tasks. Continuation training at sea or in remote locations, follow- 
ing initial attainment of a skill under formal training circumstances, could also 
be expanded in this way. 



It would be premature to assert comparable advantages for P .1. with respect 
to training in the perceptual and motor aspects of operator and workshop skills. 
Nevertheless, we have pointed out our reasons for believing that significant 
improvements in these areas are highly probable within the next few years. 
Research and development projects designed to hasten these improvements are 
as yet few in number, and exploratory in character; but gains are potentially 
rewarding enough to warrant an increased investment in them. 

The intensive research on P.I. and its many fields of application has given 
rise recently to the wider concept of a ‘technology of training’. Some of the 
component tools of this technology are already well developed. Others are 
primitive by comparison. We have noted that an adequate realisation of the 
concept entails a taxonomy of training tasks or objectives, and a classification 
of training techniques, each of them founded upon an acceptable psychological 
basis. Neither exists at the present time, however, and their construction 
presents an immediate research requirement. 

Associated with the advent of a technology of training, but more frequently 
evoked as a prerequisite of P.I., is the capacity to define terminal behaviour. 
We have referred to the critical part played in P.I. by task and behavioural 
analysis in arriving at a specification both of terminal behaviour and of the 
content of training courses. The analysis of what really needs to be learned 
can lead to benefits as great, if not greater, than improvements in instructional 
technique. (A recent Air Force study has demonstrated, after task analysis, 
that no formal training course of any kind is necessary for the task in question.) 
However, the methodology for these analyses needs to be codified and im- 
proved. We consider that the development of such a methodology for general 
Service use should also become a major research objective. 

Several new lines of research, arising out of but extending beyond the 
immediate role of P.I., have been discussed in our report. If the Services are 
to remain in the forefront of educational and training practice, they must 
encourage and assume responsibility for applied research into the improve- 
ment of current practice and the evolution of new methods designed fot future 
needs. We have emphasised the value of preserving and extending the present 
urge to develop a systematic technology of training. All work which is under- 
taken or sponsored by the Services should be in accordance with this general 
objective. Particular topics can be enumerated now as likely to repay in- 
Service effort by showing how current practice in P.I. can be improved : 

(a) Development of more suitable criteria for the assessment of programme 
effectiveness. 

(b) Evaluation of the role of an instructor in a training system based upon 
P.I. Limits of profitable application of P.I. need to be determined by 
trial and experiment. 
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(c) Determination of the relation between stringency of pacing and rate of 
effective learning, under conditions of group P.I. Unrestricted self- 
pacing may not be so rewarding as to justify the cost in administrative 
changes which would have to accompany it. 

(d) Evaluation of new and modified programming styles and devices. 

All these topics can usefully be studied in the context of actual training, by 
combining them with the developmental work associated anyway with apply- 
ing P.I. to particular training commitments. There are ait least two other sub- 
jects for research which could not be merged with field applications in the 
same way. They entail laboratory-based facilities, complete flexibility in 
experimental design, and appropriate resources for data analysis. They are: 

(a) research and development studies of computer-aided instructional sys- 
tems. (C.A.I.S.) 

(b) research and development studies of adaptive teaching machines for 
technical and operational skills. 

We regard C.A.I.S. as the most promising of all new developments arising 
from P.I. Quite apart from the probable outcomes from research which have 
been discussed earlier, a project is justified on the grounds that an experimental 
system could probably allow an accelerated production and testing of new 
programmes for P.I. Similarly, the system could be used to devise empirically 
sound instnictional procedures for particular purposes, and to show whether 
such procedures had generality for other students and subject-matters. This is 
not to say that we regard C.A.I.S. as an economic and practical proposition 
for early installation in Service Establishments. However, having noted the 
pace of development in the United States and having regard to the falling 
costs of ‘hardware’ in this field over the last year or more, we are disposed to 
believe that within a decade the C.A.I.S. could be an economic competitor to 
‘free-standing’, semi-adaptive, devices like the Autotutor or Empirical Tutor. 
We should also underline the warning that commercial development of the 
first generation of teaching machines and programmes outstripped the support- 
ing research on ‘software’. A similar mistake need not be made again if 
research on C.A.I.S. is sponsored now. 

Finally, we should point out that, although all the research and develop- 
ment work we have discussed is of direct and principal interest to the Services, 
a great deal of what is proposed has relevance to civil problems in both educa- 
tional and industrial training circumstances. The work already promoted by 
the Services on P.I. has made a unique and lasting impact upon the civil 
authorities concerned with P.I. There is no reason to doubt that future military 
studies of the subject and its derivatives can be of benefit to civil developments 
too. 
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APPENDIX 1 



Programmed Instruction in the Three Services 

This appendix summarises work on programmed instruction in the Royal 
Navy, the Army and the Royal Air Force. 



Programmed Instruction in the Royal Navy 

The ST. VINCENT Experiment 

I. A first experiment was conducted during the winter of 1961/62 in HMS 
ST. VINCENT. Its objects were to test the general claims of its protagonists 
that P.I. has significantly more to offer than conventional teaching methods. 
HMS ST. VINCENT is a shore establishment where young recruits to the Navy 
receive their basic training. The continued study of ‘school’ subjects is an 
integral part of this training. Altogether 67 junior entrants, aged 15 years 
and with an average I.Q. of 112, took part in the experiment. 



2. P.I. can be administered by the student himself under the fairly firm 
control of a teaching machine, or it can be given by means of less sophisticated 
but cheaper ‘scrambled’, or ‘programmed’, text-books. The recruits in HMS 
ST. VINCENT were divided into three equivalent groups, so that these two 
forms of self-instruction could be compared with each other and also with the 
normal system of classroom teaching. The groups were well matched with 
regard to their existing knowledge of trigonometry as well as level of attain- 
ment in arithmetic and algebra, general intelligence, reading comprehension, 
age, and educational background. A programmed course of trigonometry, 
capable of being presented in AutoTutor Mk.Il teaching machines or in 
‘scrambled book’ ( TutorText ) form, provided the common material to be 
learned. 



3. This early experiment produced encouraging results (see Table 1). Auto- 
matic instruction proved to be quite as effective as normal human instruction 
in the classroom; certainly this was true of the teaching machine, though per- 
haps not of the ‘scrambled book’. Not only was just as much trigonometry 
learned in the first place, but it was retained at least as well after an interval 
of two weeks. Persuasive, though not conclusive, evidence was revealed of a 
faster rate of learning among the recruits using teaching machines. On the 
whole, the recruits were favourably disposed towards ‘automated’ instruction, 
and a majority of them expressed a willingness to go on with the programmed 
method of learning. Even so, a preference was generally expressed for teaching 
in the conventional manner. 
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Table 1: The ST. VINCENT Experiment 



(U.S. Industries Trigonometry Programme) 





Machine 

Group 


TutorText 

Group 


Instructor 

Group 


Number of subjects 


23 


22 


22 


Matching test 


8-3 


8-4 


8*3 


Post test 


31-7 


27*2 


30-8 


Retention test 


28-6 


24-2 


26-3 


Time (to complete first 75% of 
programme) 


5*6 hours 


5-5 hours 


9'0 hours 


Time (to complete all programme) 


7'8 hours 


7*9 hours 


— 



(Note : The post test consisted of 70 questions, the first 20 of which were used for the 
matching test. AH 70 questions were again used for the retention test.) 



Experiments at HMS COLLINGWOOD 

4. The ST. VINCENT experiment naturally left many doubts about the 
practical consequences of using P.I. on a scale large enough to make a sig- 
nificant impact upon Naval training. Would the apparent success of pro- 
grammed learning be repeated during regular, serious, training for technical 
personnel? Would it persist throughout courses much longer than the 9-hours’ - 
worth of trigonometry? How formidable a task was it to programme one’s 
own material? Further experiments with these questions in view were, 
accordingly, planned and carried out in HMS COLLINGWOOD. 

5. Teaching-machine programmes were devised within the establishment to 
cover a large proportion of the basic instruction for Electrical and Radio 
Electrical Mechanics. Normal classroom teaching, for the three courses which 
were programmed, was spread over 3J weeks of Common Training, followed 
by a further 1 week for ‘Electricals’ and 4 weeks more for ‘Radio Electricals’. 

6. Matched classes of up to 20 men were instructed either by means of pro- 
grammed courses presented on teaching machines, or by the usual system of 
classroom teaching. Progress tests, written qualifying examinations, labora- 
tory work, and expressed attitudes, were all used as comparative measures. The 
teaching machines, American ‘AutoTutors’ and British ‘GrundyTutors’, were 
used under two different conditions. ‘Full automation’ was represented by 
complete withdrawal of human teaching or guidance; nor was there any direct 
supervision of working with the machines. ‘Partial automation’, as the second 
condition of P.I., meant that the normal instructional staff were on hand to 
supervise and help, if help was sought. However, the teacher’s role was 
‘passive’ and minimal so far as supplementary aid to learning was concerned. 

7. Neither of these systems of P.I. could really be judged so successful as the 
early promise of the ST. VINCENT experiment might lead one to expect. 
The existing, conventional, system was able to produce better results in the 
Common Training and Electrical courses; these results were by no means 
markedly the better, however; and the programmed course for Radio Elec- 
tricals in fact compared very well indeed with conventional instruction. Some 
quite unexpected features emerged also. Aversion to ‘machine’ tuition became 
widespread towards the end of some courses, despite early keenness. Equally 
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significant was the finding that ‘partial automation’ as defined above, had 
nothing to commend it over ‘full automation’, so far as could be judged by all 
the results. 

8. Clearly improvements were needed before sufficient cause for using pro- 
grammed instruction consistently could be demonstrated. Two sources of 
improvement seemed obvious. The programmes themselves — whose quality 
was fundamental to success — had been subjected to trial and revision before 
the experiment began; but far more time and revision than had been actually 
available should really have been permitted. Secondly, the concept of ‘full 
automation’ in training may have been premature or even fundamentally 
unsound. A positive, though more than usually economic, role for the teacher 
appeared still to be vital. 

9. Subsequent experiments in HMS COLLINGWOOD have taken matters 
to the stage where a successful integration of human and programmed instruc- 
tion seems to have been achieved. The programme for Radio Electrical 
Mechanics (REMs), although still capable of some improvement in the light 
of experience, had nevertheless already shown itself able to teach up to the 
usual standards of classroom practice. It was therefore used again, as the core 
of a new system in which active responsibility for the progress and success of 
his class was restored to the teacher. The programme, covering all basic 
instruction, extended over about 40% of the training time; this was slightly 
less machine-time than before. Practical work in the laboratory was also 
distributed over about 40% of total time, similarly representing a slight reduc- 
tion from normal. The remaining 20% was occupied by the instructor. 

10. Three definite roles were assigned to the instructor in order to ensure 
that he was fully integrated into the training system. First, he was made 
personally responsible for his students’ progress. Secondly, he was relieved 
by the programme of much of the drudgery normally associated with class- 
room teaching and laboratory work, so that he had much more time to give 
active help and encouragement to the students. Finally, he was assigned 
definite class tutorials where difficulties could be discussed, enrichment material 
given, and frequent revision carried out in an informal and helpful environ- 
ment. Although it was expected that the quality of training would be markedly 
better than that found in the first experiment at HMS COLLINGWOOD, the 
main objective of this investigation was to ‘trade’ this expected improvement 
against any economies in the employment of instructional staff. 

11. Table 2 outlines the results of this investigation. A clear-cut superiority 
is shown for integrated programmed instruction over all the other systems, 
whilst conventional programmed instruction is demonstrated to be clearly 
inferior to ordinary classroom teaching. Statistical analysis of the apparent 
differences between classes treated in the same manner have shown that the 
main outcome is unaffected; nor can the latter be attributed to prior differ- 
ences, in that the number of trainees who could be managed by one instructor 
could be doubled, representing a significant potential gain in skilled man- 
power. The questionnaire survey and the general behaviour of the trainees 
revealed practically none of the disconcerting aversion to teaching machines 
which had accompanied the earlier introduction of programmed instruction 
into naval training. 
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Table 2: HMS COLLINGWOOD Experiment Results 



System 


No. in 
Class 


Post-test 


Lab. 

marks 


Age 


1. Automated P.I. 










(a) Full i.e. no supervision 


15 


50-2 % ( 1) 


58-9% 


16/9 


(b) Partial i.e. supervision 


16 


5 1-7% (3) 


56-9% 


16/11 


All classes 




51-0% 


57-9% 


16/10 


2. Classroom Instruction 










(a) Class 1 


17 


54- 6% (4) 


49-4% 


17/5 


(b) Class 2 


19 


68-1 % ( 1 ) 


50-5% 


17/10 


(c) Class 3 


15 


55-8%(l) 


79-7% 


16/8 


All classes 




60-0% 


58-7% 


'111 4 


3. Integrated P.I. 










(a) Class 1 


15 


65-7 %(0) 


60-4% 


17/6 


(b) Class 2 


'15 


76-5 %(0) 


73-6% 


17/9 


(c) Class 3 


16 


67-9%(0) 


66-5% 


18/8 


(d) Class 4 


15 


64-2%(l) 


54-5% 


1.7/4 


(e) Class 5 


17 


70-7 %(0) 


70-2% 


18/0 


(f) Class 6 


16 


77-7%(0) 


66-4% 


17/9 


All classes 




70-5% 


65-4% 


17/10 



(Note : The figures in brackets in the post-test column indicate how many trainees 
scored less than 40%. Data for ‘all classes’ are weighted according to numbers in the 
groups.) 



12. The experimental course in electronics for Radio Electrical Mechanics 
(REMs) has continued ever since with as many as 40 teaching machines in 
use for this particular system of ‘integrated P.I’. Since no other full-scale P.I. 
application has been in operation for so long with adult students, it is of 
interest to note the outcome, in terms of average marks obtained in the 
qualifying Electronics 1 (E.l) Theory examinations. Results can be shown 
from the first 36 classes who have been instructed under the new system; each 
class is made up from 15 to 18 ratings, and two classes are handled by one 
Instructor Officer. 

Table 3: Integrated P.I. System in HMS COLLINGWOOD 



Successive Classes 
(ordinal number) 


Mean E.l 
Theory Result (%) 


Mean 

T.2 


1 — 10 


67 


111-7 


11 — 20 


66-5 


109-4 


21 — 30 


62-5 


110-4 


31 — 36 


59 


109-2 



Notes : (i) The corresponding average mark from other classes, taught conventionally 
during the period of the first 10 classes, was 63%. 

(ii) Classes 1-10 include the six ‘integrated’ classes in Table 2. 

(iii) T.2 is a measure of general intellectual standing. 
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13. There is a suggestion of less striking results recently than were originally 
obtained, so far as qualifying standards are concerned. Perhaps this is to be 
expected; certainly so, if novelty and the enthusiasm of early participants 
contributed at all to the first results. Even so, it should be remembered that 
the principal benefit, sustained throughout, has been a reduction in the 
number of Instructor Officers engaged upon the E.l phase of REM training 
from 4 to 2. When this economy is taken account of, the overall experience 
in HMS COLLINGWOOD is encouraging: distinctly so, since the system 
there has after all been one of the first (if not the first) serious applications of 
P.I. to technical training anywhere in the world. 

14. The E.l programme itself continued in use until recently in the same 
form as was prepared for the original experiments. It was always accepted 
that improvements in its quality were possible. A revised version has in fact 
been produced in HMS COLLINGWOOD, together with new printed notes 
for students and an informative ‘manual’ for instructional staff. This revised 
programme came into practical use early in 1965, and is now used for all E.l 
courses. It is particularly interesting to note that intermediate tests, and the 
E.l qualifying examination itself, have also been revised in order more closely 
to reflect the objectives of the course. The new examination has been evalu- 
ated alongside the old one. 

15. The effort required to produce an improved E.l programme was yielded 
by the savings achieved in demands upon the time of professional teachers. 
This improvement by no means exhausted the equivalent saving, however. A 
notable expansion of P.I. in HMS COLLINGWOOD has now taken place, as 
a result of more programming; which represents an investment in the promise 
of still further economies, as well as more effective training. These develop- 
ments can be enumerated as : 

(a) Linear programmes to cover the laboratory experiments and practical 
work which are an integral part of REM courses. This is a pioneering 
venture by HMS COLLINGWOOD’s staff: if as successful as early experi- 
ence of it seems to promise, this application of P.I. to laboratory work will 
have profound implications for the practical aspects of technical training. 
Four programmed laboratory experiments are in regular use for Radio 
Electrical Mechanics. They cover the Cathode Ray Oscilloscope, Series 
Resonance, Diode Characteristics, and Reactance. The programmes are in 
linear form and are presented on Grundymaster linear teaching machines, 
which are located on the laboratory workbenches. Other progammed 
experiments are being written and validated by the establishment’s staff, with 
the object of converting the whole laboratory course to the new method. 
Recently it has proved possible to save enough time on the whole electronics 
course (i.e. E.l and E.2, together with laboratory work) to accommodate 
the fault-finding programme (c). When the programmed laboratory experi- 
ments are in full use, it may be practicable to rearrange the administration 
of E.l and E.2 (see below) to accommodate individual differences in rate of 
learning. (Experience of the integrated P.I. courses for E.l theory shows 
that savings in the time for instruction could be obtained from at least a 
small proportion of ratings, provided that laboratory work were pro- 
grammed also to accommodate individual differences.) Still less supervisory 

24 

Printed image digitised by the University of Southampton Library Digitisation Unit 



staff may suffice then, since the instructional ‘load’ in laboratories will also 
be reduced. 

(b) Another branching programme, analogous to E.l in form, has been 
written and evaluated, to cover the remaining five weeks of REM instruc- 
tion in electronics (i.e. the E.2 course continued from E.l). As before, the 
production of experimental versions of the programme on film has also been 
undertaken in the establishment. Now that the programmed E.2 course has 
been added to the earlier E.l, the entire REM training load of basic 
theoretical and laboratory work can be undertaken by P.I. For the first 
time, a programmed course lasting nine weeks has been in repeated use for 
practical training. It should be noted that the ‘integrated’ form of P.I., in 
which the Instructor Officer takes a unique and critical role, has been pre- 
served. First results from this latest trial have been very satisfactory. Among 
8 classes, totalling 89 men, who underwent programmed instruction for E.2 
as well as E.l, the average mark in the final qualifying examination was 
65%, with 8 who failed to reach a qualifying mark. The average mark for 
127 men, taught by conventional classroom teaching (in 8 classes, during 
the same period of time), was 62%, with 17 who failed to qualify. Under 
the integrated form of P.I., one instructor is responsible for two concurrent 
classes. 

(c) A need for formal training in systematic fault-finding has been recog- 
nised in the establishment, and a programmed course on the subject was 
acknowledged as a good approach to meeting this need. The course consists 
principally of a new programme on the logic and general ‘strategy’ of fault- 
finding, which has been written and evaluated in preliminary trials with 
REMs. The programme takes about 3 hours to work through; it is in book 
form, with branching and linear sequences, and is used in conjunction with 
a specially adapted form of the Card Index and Key system described later 
(paras. 63-65). The programme is followed by a series of fault-finding 
exercises, also devised specially in the establishment. 

(d) It will be recalled that the very first experimental trials in HMS 
COLLINGWOOD were conducted with programmes prepared for the 
Common Training Course (3} weeks), and the first week of instruction for 
Electrical Mechanics on Motors and Generators. A, fresh approach has now 
been made, in the light of experience, to prepare P.I. courses to cover these 
commitments. (The potential gains in instructional effort are substantial in 
the case of Common Training; they are already beginning to be realised, as 
completed sections of the programme are put into effect.) 

Eight ‘chapters’ have been prepared for the Common Training pro- 
gramme; some have been evaluated and are in use, and the rest should be 
available by the early summer. The new Motors and Generators programme 
is expected to have six sections. 

Programmed Instruction at HMS RALEIGH 

16. In another Naval entry establishment, HMS RALEIGH, a concurrent 
research and development project has been in progress. This project has 
several objectives, one of which is to produce programmed courses in two 
educational subjects in which all Naval ratings must show proficiency before 
becoming eligible for advancement. Arithmetic is one of these basic subjects. 
The other is English. 

25 



Printed image digitised by the University of Southampton Library Digitisation Unit 



17. Although the Navy has leaned to some extent upon one particular form 
of P.I. (the so-called ‘branching’ method), it is by no means wedded to this 
approach. Other programming formats are being tried, and new ones will 
doubtless be developed. In HMS RALEIGH, an experimental programme on 
‘Decimals’ has been prepared in both ‘branching’ and ‘linear’ forms (these 
representing two ‘focal’ points in the spectrum of programming techniques), 
and evaluated on adult Naval recruits. The branching programme produced 
the more satisfactory learning, as it happened, also exceeding the achievement 
of classes taught normally in the classroom. Even when there was no time 
limit on instruction, the branching technique was demonstrably the more 
effective for this subject-matter which, for the majority of students, consisted 
largely of revision material. 

18. Following these experiments, a decision was taken to programme the 
entire course in Arithmetic which is received by newly-entered ratings. 

19. A first draft was ready early in 1965. It includes sections on Fractions, 
Decimals, Ratio and Proportion, Areas and Volumes, Percentages and Aver- 
ages, and Graphs. Normal instruction for this Arithmetic course, none of 
which should (in principle) be entirely new to the recruits, is distributed over 
fourteen hours of Part I training. Although conceived originally in relation 
to the requirements of the Educational Test for Leading Rate (ETLR), the 
programme has been constructed to meet the new criterion of the Naval Arith- 
metic Grading Test. Recruits are required to achieve a Grade 7 pass at the 
end of Part I training. They should also, if able enough, be given the chance 
to achieve a Grade 5 pass, which is the required standard for advancement to 
a higher Rate. More is included in the course than is necessary to achieve a 
Grade 5 Pass in the Test, however; since the programme is intended also to 
provide for students who may wish to establish an adequate knowledge before 
moving on to Higher Educational Test (HET) studies. 

20. _ The first stage of formal evaluation was completed in HMS RALEIGH 
during the summer term last year. Large classes of seamen recruits were 
instructed by means of a loose-leaf ‘scrambled book’ presentation of the draft 
programme. Several different ways of administering the programme were 
tried. The best way proved to be according to the model of integrated P.I.; 
when results in the Arithmetic Grading Tests were obtained showing an 
improvement upon ‘control’ data drawn from classes taught in the customary 
fashion. Student reactions were generally favourable to the programmed 
method. 

21. Subsequently, the programme has been slightly revised, to an extent 
deemed advisable after this first evaluation, and it has been subjected to con- 
tinuous test ever since. The Arithmetic programme, in its final form, is 
intended for use under several different circumstances, e.g. self-instruction in 
ships, as well as formal instruction ashore by integrated P.I. Therefore several 
ways of presenting the programme have been provided, and these have been 
subjected to experimental trials in HMS RALEIGH under each of several 
degrees of supervision. Since large classes, up to 50 in size (about double the 
usual numbers) have been the responsibility of a single Instructor Officer dur- 
ing these trials, it is evident that some gains in instructional productivity have 
already been effected, even under experimental conditions. 
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22. Trials of the programme have been completed with a teaching machine 
presentation (AutoTutors and GrundyTutors). Average rises, from the ‘pre- 
programme’ to the ‘post-programme’ Arithmetic scores, corresponded to an 
improvement from Grade 7 to Grade 5. Performance was at least as good as 
after conventional instruction, even when the programme was experienced as 
self-instruction with no more than passive supervision; it was best of all under 
the condition of integrated P.I., when the instructor participated actively at his 
discretion. Also under trial during the last six months have been two non- 
mechanical presentations of the programme, using methods derived from the 
Card Index and Key System (see para. 63, ahead). Some difficulties have been 
met in the preparation and administration of materials for these presentations; 
but preliminary results suggest that a programme which is a kind of ‘scrambled- 
book, plus routeing-card’, and which preserves a record of the students’ 
responses, may be as effective as the mechanical presentation. 

Programmes for HET 

23. The main obstacle facing a widespread use of P.I. is to find the time and 
expertise for programme writing. In principle, the best procedure is to use 
professional teachers who have been properly introduced to the theory and 
practice of programming. They are then assigned to prepare courses in the 
particular subject-matters they know best. Nevertheless, in view of the scarce 
resources in programming staff, there may be some commercial courses which 
are relevant enough to be worth trying in a Naval context. 

24. One such course appeared to be the ‘Ordinary’ level course in General 
Mathematics produced by United States Industries, Inc. (Great Britain) Ltd. 
Without attempting to evaluate this course (in the technical sense of ‘evalua- 
tion’), a qualitative appraisal of the programme’s content and style at least 
enables its relevance to self-instruction on the GCE/HET syllabus to be judged. 
Only if its general coverage and difficulty-level were appropriate for Naval 
purposes, would a more objective study of its effectiveness be worthwhile. Such 
an appraisal has been carried out in HMS GANGES. It is based upon the 
experience of selecting three luniors, who would otherwise not have obtained 
the opportunity of studying in the establishment for a GCE examination, and 
allowing them to work unsupervised through the programme. The learning 
situation was nevertheless carefully regulated throughout the 52 hours of term- 
time allowed for self-instruction, using an AutoTutor machine. 

25. From the detailed records of progress, test and examination results, and 
comments of the students, it was concluded that : 

(a) although much of the programme was relevant to Naval HET require- 
ments, a substantial portion was not. Nor was all the Service syllabus 
covered. Even though the style and presentation were judged to be generally 
good, the programme as it stood was therefore not acceptable for practical 
use. 

(b) self-instruction with teaching machines (such as the AutoTutor used for 
this appraisal) was thought by the establishment’s staff to be uneconomic for 
GCE or HET studies in a shore establishment like HMS GANGES. Given a 
thoroughly relevant programme, however, modelled on the lines of the U.S. 
Industries course, programmed instruction at sea could serve a most useful 
purpose in catering for the voluntary student. 
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Programmed Instruction at Sea 

26. The programme which has just been mentioned is one of several Naval 
and commercial programmes which were retained in HMS CENTAUR for a 
year, together with two AutoTutors. No formal schedule of trials could have 
been carried out in an operational carrier. Nor were the various programmes 
chosen for reasons other than their availability at the time and the likelihood 
that they might serve the interests of ratings engaged in voluntary technical or 
GCE studies. 

27. Nevertheless, preliminary experience of P.I. at sea was obtained, even 
though the ship concerned and the circumstances of its commission in the Far 
East may not have offered characteristic seagoing conditions. Reports from 
HMS CENTAUR were informative, for example, about snags which must be 
overcome before P.I. can fulfil the useful role at sea which it ought to occupy. 
More sea trials, perhaps in smaller ships, are needed before firm conclusions are 
drawn, and several limited trials at sea have subsequently been completed. In 
the meantime, experience in HMS CENTAUR led to these conclusions : 

(a) Provided that programmes are seen to be strictly relevant to the aspira- 
tions and interests of seaborne personnel (and this proviso is critical) potential 
students will welcome and use them. Free time for voluntary study may be 
severely limited in operational ships, however. 

(b) To make the fullest use of this time (and also, possibly, to capitalise for 
the purposes of private study upon periods of stand-by, ‘waiting’, duty), it may 
be necessary to forgo some of the advantages of machines, which tend to be 
static and not immediately accessible. Portable, non-mechanical, ways of 
presenting programmes may be more practicable than machines, at sea. 

(c) Nevertheless, wherever a classroom is permanently fitted-up so that 
formal instruction can take place at sea, teaching machines should take their 
place along with other facilities. Their design requires fresh attention, how- 
ever, in order to minimize size, and the need for special stowage and main- 
tenance. 

Training by P.I. for Operational Skills 

28. The principles of learning upon which P.I. is founded are not exclusive to 
academic subject-matters, nor to drills and procedures. They are valid also for 
manual and perceptual skills. Indeed, programming techniques developed as a 
direct outcome of laboratory studies of how such skills are acquired. Neverthe- 
less, there have been few serious attempts to use programmed teaching 
machines for the kinds of skill encountered in the handling of surveillance and 
weapon systems. 

29. At the Applied Psychology Unit, Teddington, the last two years have seen 
the development of a fresh approach to two different problems of sonar operator 
training. In the particular cases which are under investigation, formal training 
by ‘conventional’ means was insufficient: partly due to the newness of the equip- 
ments and partly to the lack of detailed understanding of how the operating 
skills themselves were composed. The Unit’s attempt to programme, and auto- 
mate, the basic instruction has therefore been preceded by an analysis of these 
active and passive classification skills. 
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30. Both of these operating tasks require a general knowledge and under- 
standing of what is happening to produce the various visual and auditory 
displays of information. Conceptual as well as perceptual components go to 
make up the required skill of an operator. One teaching machine which 
appeared at first to be well suited to the control of programmes for such a 
combination had to be discarded, due to its unreliability and lack of flexibility. 
Instead, the Unit is now engaged in preparing programmes for a modified 
version of the Empirical Tutor system, which accommodates visual and audi- 
tory presentation and either constructed or multiple-choice responses. Several 
models of the modified Tutor are being installed in a mobile ‘classroom’, ready 
for experimental trials during the present year. 

31. It is hoped that this P.I. application will become a satisfactory prototype 
of automated training equipments for a variety of modem seaborne sonars. 
The experimental version is certainly promising enough to have initiated a 
second attempt to apply the technique to a special training problem which 
stems from a partly conceptual, partly perceptual, skill. An Empirical Tutor 
has been obtained for the establishment which is developing shore-based train- 
ing equipment for computer-aided action information systems. The object is 
to link the Tutor to a simulator of various data displayed within the system, 
and to programme introductory exercises to familiarise personnel with alpha- 
numeric tabular presentations and their association with plan-position data. 



Choosing a Programming Style and Teaching Machine 

32. One of the practical problems which faces any programme writer is to 
decide which of the various programming techniques to adopt. Another is to 
choose the method of presentation: and, if a mechanised one, which particular 
machine. No clear-cut rules of decision-making can be offered; these problems 
can only be solved with complete satisfaction by the minority whose theoretical 
predilections permit them no latitude at all. 

33. In practice intuitive and economic criteria will have to be employed in 
most of the programming situations encountered in the Services. However, the 
more empirical and comparative evidence there is, the more informed and 
confident can decisions be; hence it is useful to accumulate relevant evidence 
whenever the circumstances permit. Such an occasion occurred during 1963 in 
HMS RALEIGH, where it was decided that the programme on Decimals — 
though shorter than one would ideally choose as a reliable exemplar of P.I. 
— was nevertheless so well ‘standardised’ for its teaching merits as to make it 
a useful instrument for comparing different methods of presentation. 

34. The objective was to allow groups of recruits who were matched on T2 
scores to work through the standardised Decimals programme, in one of eight 
different ways of presenting it. The programme was in linear or branching 
form. Equivalent linear versions were prepared for use with the Grundymaster 
machine (International Tutor Machines) and the S.S.L. Programmed Teaching 
Machine (Scientific Systems Ltd.). Branching versions were prepared for the 
U.S.I. AutoTutor and I.T.M. GrundyTutor, and also for the S.S.L. Machine 
which is designed to accommodate both kinds of programme. 
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35. The experimental plan included provision for such variation in the con- 
ditions of learning as to allow eight possible conditions. In the event, the data 
derived from two of these conditions proved to be unreliable, and they are 
therefore excluded from this summary. In the following table are shown the 
average class gains in marks on the criterion test of proficiency at Decimals, 
obtained as a result of P.I. in one or another of the six conditions. Since the 
actual programmes were unchanged from those used in earlier studies at HMS 
RALEIGH, and the learning time was the same, it is instructive to look at 
these more recent results in comparison with what was found before. The last 
two linear and branching entries in the table indicate the earlier findings. 



Table 4: Average Learning Scores from Various Forms of P.I. 



Programme 

Style 


Form of 
Presentation 


Learning 
Time Allowed 
(minutes) 


Learning 
Score (Gain 
in marks) 


T.2 


Branching 


Auto /Grundy Tutors 
(Integrated P.I.) 


120 


8-00 


92 


Branching 


S.S.L. Machine 


120 


6*25 


93 


Branching 


S.S.L. Machine 


120 


7-00 


88 


Branching 


Loose-leaf 
Scrambled booklet 


120 


7-45 


95 


Branching 


Loose-leaf 
Scrambled booklet 


250 


7-30 


90 


Linear 


S.S.L. Machine 


120 


6-10 


94 


Linear 


Grundymaster 


120 


4-90 


90 


Linear 


Grundymaster 


120 


6-05 


89 


Linear 


Numbered Cards 


120 


5*05 


91 


Unear 


Numbered Cards 


250 


5-50 


83 



Notes on Table 4 



(i) The two S.S.L. Branching groups were Seamen and Engineering Mechanics 
respectively. All others were Seamen. 

(ii) The two Grundymaster groups learned under different conditions of machine 
presentation, one of which permitted ‘confirmation’ of response sometimes before 
the response was made. 

(iii) A formal test of the reliability of these differences among groups has not been 
made. The trends can only be taken as suggestive, therefore. 

(iv) Comparable learning scores from normal classroom teaching averaged 5-50 (i.e. 
for the same period, and subject-matter, with matched classes). 

Too much should not be made of these results in the present context, except to 
say that : 

(a) the decision to create a branching programme for the Naval Arith- 
metic programme is supported by the data. 

(b) the ‘integrated’ system of using P.I. (i.e. with an active but new role 
for the teacher) appears qualitatively the most effective; although the 
statistical reliability of this outcome is uncertain. 

(c) the advantages of machines over non-mechanical presentations are not 
displayed in these data so far as the branching programmes are concerned; 
nor is there more than marginal support for their superiority for linear 
programmes. An important qualification to note, however, is that the 
programme was brief, by practical training standards (two hours). 
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Conclusions from Naval Studies 

36. The experience which Naval and other Service establishments have 
accumulated so far strengthens the conviction that P.I. represents a major 
technological ‘breakthrough’. Its stress upon the clear definition of training 
objectives and measurement of effectiveness are representative of its claim to 
offer a complete system of training, not just a novel technique. Refinements in 
programme writing, and in methods of presentation and control, have multi- 
plied during the last year or two. In the light of this progress, and from our 
own Naval experience of experimental applications, it is probably no exagger- 
ation to claim that the promising improvements which are already fore- 
shadowed could be greatly exceeded in future. 

37. The scope as well as the quality of P.I. is increasing with each new attempt 
to apply the basic technique. For example, agreement has been reached with 
the establishment concerned (HMS FISGARD) to study the application of 
P.I. principles and technique to several subjects taught to skilled craftsmen. 
P.I. for workshop technology, and also for the actual learning of machine-tool 
operations, are among the possibilities to be tried. Also at HMS FISGARD, 
a new style of programming (based on the Pressey ‘auto-adjunct’ programme) 
is being drafted to cover the requirements of a Science syllabus for apprentices 
undergoing accelerated training. Several other training establishments, and 
also the Royal Naval College, Greenwich, have considered how to exploit P.I. ; 
and programmes on special topics have already begun to be written in some 
of these establishments. 

38. However, the practical gains which the Navy seeks are severely limited by 
the amount of programming effort which can be invested. The modest 
economies so far effected have been profitably diverted to a small expansion 
of programming; but the ultimate benefits would seem to justify a bigger and 
more formally organised Programmed Instruction Unit; which could assume 
responsibility for writing Naval programmes, evaluating them, and advising 
other establishments on particular applications. If it were possible to launch 
a formal Programming Unit, with time for advisory work in other establish- 
ments, there is every likelihood that a number of officers involved in training 
may be keen to undertake limited programming ventures themselves, under 
guidance. Many short courses of instruction in specifically Naval topics 
(courses in which the informational content is presented, say, in one or two 
hours only) are ‘expensive’ in the demands they make upon instructional staff. 
Programmes, with or without the control of teaching machines as circum- 
stances warrant, may confer substantial benefits here. (Detailed proposals for 
setting up such a Unit have subsequently been accepted in principle; it is 
expected that a firm start will be made in the near future.) 

Programmed Instruction in the Army 

Programme Writing 

39. The Army’s major efforts in programme writing have taken place in four 
establishments; 

(a) The Royal Military College of Science, Shrivenham; 

(b) The Army School of Education, Beaconsfield; 

(c) The Royal Army Pay Corps Training Centre, Worthy Down; 

(d) Alexandra Grammar School, Singapore. 
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The Royal Military School of Science, Siiriveniiam 

40. Two programmes were produced by members of the College Staff at the 
level of a first degree in electrical engineering. The subject matter covered was 
8 of 77 lectures on electronic engineering in a course preparing officers for 
technical staff appointments. The programmes deal with Triode Amplifiers, 
and Emission and Conduction (each topic being normally dealt with in four 
lectures). 

41. The programmes were presented in books containing white pages (which 
constituted the prime path), each with corresponding red pages giving further 
explanation and, sometimes, corresponding blue pages giving even more 
fundamental help. The pages were not scrambled, each white page being 
followed by its corresponding red and (sometimes), blue page. The format 
was thus skip-branching with two or three levels of difficulty, students by- 
passing one or two coloured frames according to their responses. Sometimes 
multiple-choice responses were required, sometimes written responses. The 
student is, of course, his own comparator in this situation and he decides 
whether he should read the coloured remedial frames or not. 

42. The programmes in their final form (after a trial with laboratory assist- 
ants as subjects) consisted of: 

Triode Amplifiers : 55 white frames, 55 pink frames and 50 blue frames. 
Emission and Conduction : 69 white frames and 69 pink frames. 

A comparison of these programmes with conventional lecturing is referred to 
below. 



The Army School of Education, Beaconsfield 

43. Aside from several short programmes, some of which have been produced 
as illustrative material for programming courses, the School has undertaken 
two programmes. The first version of a programme in military law is now 
complete and has undergone validation. The other programme is a fairly 
lengthy one which will cover the map reading syllabus for the Army’s First 
and Second Class Certificates of Education. Both programmes have been pro- 
duced on a part-time basis by RAEC officers serving at the School. 

44. The programme on military law consists of 170 frames. It is a branching 
programme, the prime path frames having, for the most part, three-choice 
questions. It aims to teach officers the basic military law they would need 
within a unit. There are three parts. Part 1 deals with recognition of offences 
against the Army Act 1955, the circumstances under which it is necessary to 
arrest an offender, and procedure on arrest. Part 2 deals with compiling charge 
reports and the hearing of charges. Part 3 deals with punishments that can be 
awarded by subordinate commanders and by commanding officers. Each part 
is followed by a test and there is also a final test. 

45. The programme exists in three formats:— Scrambled Book, Film (for 
AutoTutor or GrundyTutor), and Card Index and Key. (Reference is made to 
this last format below.) 
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46. The map reading programme is mostly linear, requiring written responses, 
though one of the sections so far completed has been written in the multiple- 
choice branching style. It is to be used along with Sheet 80 of the One Inch/ 
One Mile Ordnance Survey (7th Series). The programme for the 2nd Class 
Certificate will consist in its final form of the following sections : 



Conventional Signs 
Four Figure Grid References 
Six Figure Grid References 
Further Conventional Signs 
Scales: 

Representative Fractions, 
Protractor Scales and 
Scale Lines 
Relief 
Direction 



Grid Bearings 
Magnetic Bearings 
Intersection 
Resection 
Gradient 
Intervisibility 
Description of Routes 
Time and distance 
Route Cards 



The Royal Army Pay Corps Training Centre, Worthy Down 

47. A 600 frame branching programme has been written covering the intro- 
duction to regimental fund accounting which is normally taught in the class- 
room at the beginning of courses run by the Centre. It has been produced in 
spare time between courses by one of the Centre’s instructors. This introduction 
consists of the basic book-keeping principles most relevant to regimental 
accounting procedures, namely : 

Double entry recording of transactions; 

Classification of transactions as ‘Capital’ or ‘Revenue’; 

Classification of accounts as ‘Personal’, ‘Real’, or ‘Nominal’; 

Opening and closing accounts; 

Preparing profit and loss account and balance sheet; 

The nature of assets and liabilities, ‘fixed’ and ‘liquid’ assets. 

The programme is, mostly, either simple ‘herring-bone’ or ‘skip-branching’. 
There are, however, several more complicated sequences as well as a built-in 
test with wash-back loops graduated according to performance on the test. An 
evaluation of the first version of the programme (see below) has been carried 
out. The programme’s present format is 8" x 5" cards housed in ‘Roneodex’ 
wallets. 



Alexandra Grammar School, Singapore 

48. This school is for the families of Service personnel. Three written response 
linear programmes have been written by the staff on a part-time basis. The 
length of these programmes in terms of frames is not known but they provided 
for an average attendance of twelve 40 minute periods in the case of a chemistry 
programme (writing formulae, writing and balancing simple equations), for an 
average of 13 periods of biology (digestion in the human body), and for 
24 periods of French (“Nos Voisins,” 7 to 10). A report on this work states 
that, “at a conservative estimate”, 36 hours of programming time were required 
for each hour of study. 

49. Before administration these programmes were tried out on top stream 
secondary modern school subjects and “revised to minimize error rate”. Pre- 
sentation of the programmes was by Grolier Minmax machines. The results of 
an evaluation of this work are given in the next section. 
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Comparisons with Existing Instruction 



50. Various comparative studies of programmed instruction have been 
attempted. 



School of Electronic Engineering, Arborfield 

51. Perhaps the first comparison of programmed instruction with existing 
methods in the Army was at the School of Electronic Engineering in May- 
June, 1962. The programme used was part of the U.S. Industries basic elec- 
tronics TutorFilm and was presented to REME craftsmen by AutoTutors. 
The report on this work concluded, inter alia, that the programme taught as 
well as a good artificer instructor who took a control group through the same 
subject matter, but it also points out that, despite attempts to match the groups, 
it subsequently turned out that “the division of ability was not in fact equal’’. 



The Royal Military College of Science, Shrivenham 

52. In the 1963 Lent Term the College compared the effectiveness of the pro- 
grammes in Triode Amplifiers and Emission and Conduction described above, 
with conventional lecturing. Thirty-odd officers in their late twenties or early 
thirties were divided into two groups. ‘Group A’ was assigned to supervised 
study of the programmed text on Emission and Conduction, ‘Group B’ attended 
four lectures, each of which was supplemented by a sheet of problems for 
private study. Time spent on lectures, on the programmed text and on all 
private study was recorded for each officer. Following a test on Emission and 
Conduction the methods were reversed between groups, i.e. Group B was 
assigned to the programmed text on Triode Amplifiers and Group A to lec- 
tures. This second half of the experiment was similarly followed by a test on 
Triode Amplifiers. Thus, in this interesting ‘cross-over’ experimental design, 
each subject is his own control — a feature which one might wish to see more 
often in experiments with programmed learning. 



53. However, with regard to the ability of the two groups to assimilate two 
different subject matters, the matching data cited in the report of this experi- 
ment is rather limited, each officer being rated on a three point scale of previ- 
ous knowledge. The report concludes, inter alia, that in terms of time and 
achievement on post tests, the programmed texts were comparable to conven- 
tional lectures. 



The Royal Army Pay Corps Training Centre, Worthy Down 

54. An evaluation of the programme on regimental fund accounting described 
above was carried out by the Army Operational Research Establishment in 
April, 1964. The subjects were taken from two groups — apprentices and re- 
cruits. The classroom instruction was undertaken by an experienced instructor, 
possibly of above average ability. The results of various pre-tests and post- 
tests are summarised in Table 5. 
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Table 5: Regimental Fund Accounting 
Pre- and Post-Test Means 







Programmed 

Instruction 


Classroom teaching by 
RAPC Instructor 






A p prentices 
(n = 6) 


Recruits 
(n = 6) 


A pprentices 
(n = 6) 


Recruits 
(n = 6) 


Pre-Tests 

& ! b i •» 


Raven’s 

Matrices 

(1938) 


47-7 


49-7 


46-8 


48-2 


Accounting 
-Principles 
(Multiple 
Choice Test) 


17*7 


6-7 


16-5 


7-0 




Accounting 
-Principles 
(repeat of 
Pre- test) 


26’ 8 


25-3 


24-0 


27-1 


Post -Tests 

r 


Accounting 

-Definitions 

(written 

response) 


27-0 


31-0 


21-8 


31-8 


in irifi 

;] J |j , 


Accounting 

-Practical 


61-5 


58-6 


42-1 


70-6 



55. It will be noted that there is an apparent interaction between the two 
groups and the two methods of instruction, apprentices tending to score higher 
under programmed instruction, recruits higher under conventional classroom 
instruction by an RAPC instructor. However, only one of these apparent 
differences, that between apprentices on the practical accounting test, is statis- 
tically significant. 

Alexandra Grammar School, Singapore 

56. The programmes written by the School staff and described above were 
evaluated, together with a commercial algebra programme, on first form stud- 
etts in the latter half of their second term, i.e. March- April, 1964. (The com- 
mercial programme was TMI Grolier Algebra I, i.e. simple equations.) Thirty- 
six pairs of students, matched in terms of sex, intelligence and attainment in 
examinations at the end of their first term (rank order) were divided into con- 
ventional classroom instruction and programmed instruction groups. 

57. Table 6 shows the comparative achievements and attendance of 32 of 
the 36 pairs for whom data was complete. Three of the differences between 
methods are significant, two in favour of programmed instruction (Biology 
and Chemistry), and one in favour of conventional classroom teaching 
(Algebra). 
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Table 6: Alexandra Grammar School. Post-Test Means, and Average 
Attendance of the two Matched Groups (N — 32 in each Case) 



Subject 


Programmed 

Instruction 


Conventional 

Classroom 

Instruction 


Average Attendance 
(40 minute periods) 


P.I. Group 


C.C.I. Group 


French 


37 


34 


24 


26 


Algebra 


41 


47 


14 


13 


Biology 


48 


31 


13 


12 


Chemistry 


77 


54 


12 


12 



Other Comparisons of Programmed Instruction with Conventional 
Classroom Instruction 

58. An experiment was carried out by the Army Operational Research Estab- 
lishment at the Army Apprentice School, Arborfield, in October and Novem- 
ber, 1963. The main object of this experiment was to compare presentation of a 
trigonometry programme by AutoTutors with the same programme presented 
via a new and fairly inexpensive device, of which more is said below. How- 
ever, the experiment had a secondary objective — to compare programmed 
instruction with instruction by an experienced RAEC officer. This officer 
taught two groups, one twice as large as the other. Paradoxically, the smaller 
group had the lower post-test score, though the difference between the two 
groups and, indeed, the differences between all the groups (programmed 
instruction and conventional classroom instruction) is statistically non-signifi- 
cant. Table 7 summarises the matching test and criterion test data for the four 
groups in this experiment. 



Table 7: Arborfield Apprentice School. Pre- and Post-Test Means 
(U.S. Industries Trigonometry Programme) 



Group 


n 


AM .4 
(General 
Intelli- 
gence) 


N.S.6 

(Reading 

Compre- 

hension) 


Arith./ 

Algebra 


Geom./ 

Trig. 


Trig. 

Post- 

Test 


AutoTutor 


16 


80-63 


51-25 


12-15 


6-94 


28-63 


Card Index 
and Key 


17 


79-82 


51-94 


13-29 


7-82 


28-12 


Classes taught 


14 


80-21 


52-57 


14-21 


7-79 


29-71 


by 

RAEC Officer 


6 


80-17 


50-83 


14-17 


8-00 


25-17 
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59. More recently the ASE programme on Military Law has been evaluated 
on RAEC Officers and RAPC Officers and Warrant Officers. The main find- 
ings, which indicate that the programme is probably more efficient than con- 
ventional classroom instruction, are given in Table 8. Again the Card Index 
and Key System was used as well as a teaching machine (the GrundyTutor). 



Table 8: Means for Comparative and Criterion Tests — 
Military Law Programme 





R.A .E.C. Subjects 


R.A.P.C. Subjects 


Variable 


Grundy 

Tutor 


Card Index 
and Key 


Grundy 

Tutor 


Conventional 

Classroom 

Instruction 




(n = 15) 


(n = 15) 


(n = 12) 


(n = 10) 


AH4 (General 
Intelligence) 


97-80 


94-40 


92-08 


81-40 


Black’s Test 
(Reading 
Comprehension) 


41-93 


46-73 


34-08 


30-70 


Military Law 
Pre-Test 


43-93 


42-20 


53-58 


38-50 


Age - months 


365 


399 


526 


471 


Military Law (a) 


65-93 


67-33 


70-58 


60-40 


Post-Test (b) 


66-35 


66-35 


66-35 


66-35 


Overall Time (a) 


96 


150 


132 


265 


(Minutes) (b) 


95 


154 


128 


246 



(a) observed mean. (b) observed mean adjusted to take account of initial differences 
between groups. 



60. Finally, an evaluation was undertaken by AORE of the U.S. Industries 
Slide Rule programme. Subjects in the experiment were officer cadets in their 
second term at the Royal Military Academy, Sandhurst. Interpretation of the 
data is complicated by important differences between the content of the pro- 
gramme and R.M.A., Sandhurst, teaching of the slide rule. The programme 
produced unfavourable reactions to P.I. in a majority of the subjects. 

An Inexpensive System for Branching Programmes 

61. Those responsible for armed services education and training are fre- 
quently faced with groups which are heterogeneous with respect to intelligence, 
existing relevant knowledge and skills, etc. ‘Branching’ devices like the Auto- 
Tutor are attractive because of their capacity to adapt the material presented to 
such individual differences. Such devices are also very expensive. 
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62. The early work in the Navy demonstrated that a book format such as 
the TutorText is no substitute for a machine like the AutoTutor. Experiments 
have been carried out by AORE with an alternative system for branching 
programmes. This system is fairly cheap and has many of the advantages of 
the machine as well as some of its own. 

63. The system has two components: — a card index containing the instruc- 
tional information — the ‘frames’; and a key which indicates the route through 
the programme to be taken by the student. The key in our experiments con- 
sisted of a card on which was printed a matrix. The rows of the matrix 
correspond to the frames in the programme, the columns to alternative re- 
sponses. The student responds to the programme by shading appropriate cells 
in the matrix with a soft pencil. The effect of shading is to reveal the image 
of a number which is embossed in the Card but hidden by the top layers of 
paper. This is the number of the frame which should next be studied. 

64. It is clear from Tables 7 and 8 that the attainment of matched groups 
with this system is as good as with the AutoTutor and GrundyTutor machines 
containing the same programme on films, with one proviso. When, as in the 
evaluation of the Military Law programme, the length of a session for each 
student is determined by the time he takes, working at his own pace, to reach a 
previously fixed point in the programme, then the Card Index and Key Sys- 
tem takes longer than the machine. 

65. The system has several advantages and disadvantages vis-a-vis the 
machine which were not all obvious until it had been used experimentally. To 
summarise these, there are three main disadvantages having to do with: 

(a) Cheating. The Card Index and Key System is cheat-revealing rather 
than cheat-proof. The machine can prevent undesirable back-tracking (e.g. 
looking up the answers to ‘built-in’ test questions). This the system cannot 
prevent, but only make difficult. However, to the extent that filmed pro- 
grammes depend on the machine’s error counter, machines are strictly only 
cheat-revealing devices. 

(b) Storage and Capacity. Programmes stored on cards will obviously 
make far greater demands on storage space than will programmes stored on 
film. Similarly, the capacity of the machine greatly exceeds that of inexpen- 
sive card indexes. 

(c) Changing the Programme. This is much less convenient than with the 
machine. 

The advantages may be summarised as follows: 

(1) Low Cost. 

(2) No power requirement. 

(3) Random Access. A path can go from any one frame to any other 
frame. No machine could achieve random access so inexpensively. 

(4) Programme Revision. The record of choices left by each student on his 
key provides data for programme revision; i.e. data on which frames 
give general difficulty. This information is not provided by the machine. 
Furthermore, replacing frames or whole sequences is easy with a card 
index. Filming positively discourages continuous modification of the 
programme (an ideal advocated by many and practised by fewl. 
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(5) Human Teacher Participation. A combination of human teacher and 
programmed instructional device is probably superior to either alone 
and the complete absence of human support is likely ito prove un- 
popular. Because the system is cheap, the problem of making economies 
in teaching staff is avoided. However, the whole question of the role of 
the teacher in a class under programmed instruction is deserving of 
further study. Presumably, if the teacher is to participate and not 
merely invigilate, he needs feed-back of information on each of his 
pupil’s movements through the programme. Apart from telling him 
the frame reached, machines only indicate (and not always reliably) 
the number of ‘errors’ so far made. What the human teacher most 
needs to know is not just how many, but which errors have been made, 
which concepts have given a particular pupil difficulty. The record of 
choices left by each pupil in his use of the key goes some way towards 
providing this information. 

66. In conclusion, the system may be a useful compromise for newcomers to 
the field of programmed learning because it permits flexible, even sophisticated, 
programming without committing them to much expense. Furthermore, the 
inevitable faults of the programming novice can conveniently be put right. 

Further Research 

67. In the future, an investigation is planned into the contribution of 
‘branches’ to learning. The programme to be used will be Thornhill’s “Water- 
loo Campaign”. 

68. It is also hoped to study adjunctive programming, possibly using the 
Common Core electronics texts, with and without the supporting control of 
tests indicating specific sections of text for revision. 

Conclusions Drawn from Army Work 

69. Glittering prizes are offered to those who introduce programmed instruc- 
tion into educational and training situations. There has been no shortage of 
good reasons why programmed instruction must be, in principle, superior to 
conventional classroom instruction, and indeed the first empirical studies in 
the armed services seemed to promise dramatic improvements in learning rate. 
More recent work, however, has tended to indicate (as do most of the fore- 
going comparisons in Army establishments), something nearer to parity be- 
tween programmed instruction, on the one hand, and conventional classroom 
instruction on the other. 

70. This tendency may, of course, have many contributory factors and it is 
probable that some of these are, as yet, little understood. However, two 
possibilities immediately suggest themselves. As comparative studies become 
more numerous, there may be a tendency for teachers of the control groups in 
such studies to be more effective, either because teachers of above average 
ability are deliberately sought to set programmed instruction a rigorous test, 
or because the teachers, understandably, engage in herculean efforts to beat 
the programme. To the extent that this is true the findings of recent studies are 
grounds for optimism rather than disenchantment. 
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71. A second possible cause for the recent findings of experiments with pro- 
grammed instruction is more disturbing. It is that the standard of programming 
currently being achieved is failing to exploit to the full the possibilities of 
programmed learning. It will be clear from the foregoing account that such 
programming as is being undertaken in the Army is usually on a part-time 
basis and is frequently in addition to regular duties. It is very likely that time 
constraints may be leading programmers, whose task can be agonising in the 
best working conditions, to neglect the internal validation of programmes; i.e. 
try-outs with individuals from the target population tend to be perfunctory, 
and drastic revision of programmes rare. As a result, programmes are probably 
too teacher- or instructor-orientated; and there may be important discrepancies 
between an approach based on the teacher or instructor’s preconceptions and 
a truly student-centred approach, as has been shown by Mager (1961) in the 
case of sequencing. With regard to programming style, efforts have almost all 
crystallised, possibly prematurely, into either multiple choice branching pro- 
grammes, or constructed response linear programmes. With few exceptions 
the style chosen seems to depend on the programme writer’s predilections, 
rather than, e.g. on the responses specified in the terminal behaviour, or on 
the degree of heterogeneity of the target population. 

72. Finally, more extended studies of programmed learning have yet to be 
carried out in the Army. Most programmes so far completed have been fairly 
short. The Army School of Education’s programme on map reading will prob- 
ably be the first long programme to be introduced. It is only by the introduc- 
tion of such longer programmes that the problems of integrating programmed 
instruction into Army educational and training situations will be solved. 

Programmed Instruction in the Royal Air Force * 

73. Initial investigations were carried out by the Royal Air Force School of 
Education and the Research Section at Headquarters, Technical Training Com- 
mand. In order to prevent any duplication of effort, the School of Education 
agreed to confine its efforts to the problems of programme writing, whilst 
Technical Training Command Research Section evaluated the application of 
programmed learning and teaching machines in some of the problem areas of 
technical training. 

The Yatesbury Experiment 

74. The first evaluation was carried out at No. 1 Radio School, Royal Air 
Force, Yatesbury, in the Autumn of 1961, using the branching programme on 
Trigonometry written by Norman Crowder and Grace Martin. Twenty-nine 
students were taught by means of teaching machines, and a control group of 
thirty students of comparable aptitude and educational attainment were taught 
by conventional classroom methods. At the end of the experiment, it was 
found that there was no real difference in the achievement of the machine- 
taught group and the classroom group — except that the teaching machine 
group reached their standard in only 39% of the time. A retention test was 
given three months after the completion of the experiment, and no significant 
difference was found between the scores of the two groups (see Table 9). 

* This section is partly based on a note describins Royal Air Force experience with 
P.T. written by Squadron Leader I. K. Davies, R.A.F. (Davies, 1964c.) 
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Table 9: Royal Air Force Yatesbury Experiment 
Criterion Scores for Machine and Classroom Groups 





AutoTutor Group 


Classroom Group 


Post- 

test 


Retention 

Test 


Time 


Post- 

test 




Time 


Formal 

Study 


Formal 
Plus Private 
Study 


Test 


Formal 

Study 


Formal 
Plus Private 
Study 


Mean 


59% 


68% 


5:30 


5:53 


58% 


66% 


11:00 


14:23 


N 


29 


22 


29 


29 


30 


23 


30 


30 



The Melksham Experiment 

75. Since the Trigonometry programme used at Yatesbury was relatively 
short, the results of this experiment could hardly be generalised to most RAF 
training courses, and it was therefore decided to evaluate teaching machines 
over a longer period of instruction. Accordingly, a further experiment was 
carried out at No. 12 School of Technical Training, Royal Air Force, Melks- 
ham, in 1963. A battery of AutoTutors was used to teach sixty-two Electrical 
and Instrument Fitters the fundamentals of electronics. The portion of the 
syllabus which was used in the experiment extended over six weeks, and 
covered DC and AC Theory. During this time about half of the students’ time 
was spent in the classroom, while the remaining time was either spent on 
laboratory work or in the workshops. However, the programme only covered 
the classroom instruction phase, so that the laboratory and workshop instruc- 
tion was carried out in the normal manner. 

76. The machine group was matched for attainment in mathematics and 
science with a conventionally taught group of fifty-three students. A blind 
control of seventy-six students — who were unaware that they were taking part 
in the experiment — was used for comparison purposes. The results are 
summarised in Table 10. It will be seen that there was no real difference 
between the classroom-taught group and the machine-taught group as far as 
total learning time and retention scores were concerned, but that there was a 
real difference in the time spent on private study and in the post-test scores. 
The post-test scores suggest that as far as actual attainment was concerned, 
the classroom group was superior in their grasp of the factual content of the 
examination. In a test for understanding and depth of knowledge, however, 
the machine group was significantly better. 



Table 10: Royal Air Force Melksham Experiment 1963 





No. 


Matching 
Test (Maths, 
and Science) 


Post-tests 


Total Time 


Retention 

Test 


Time spent 
in private 
study 


Knowledge 


Depth 


Classroom 

Group 


53 


64% 


75% 


53% 


77 hours 


57% 


28 hours 


Machine 

Group 


62 


64% 


67% 


68% 


75 hours 

(68 hours without 
revision) 


59% 


16 hours 


Blind 

Control 


76 


65% 


78% 


56% 


- 


- 


29 hours 
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77. The results are not so dramatic as those obtained at Yatesbury, but a 
part of the explanation is contained in the differences between the programme 
and the normal school syllabus. At least three questions in the first post-test 
(i.e. equivalent to 10% of the marks) were not taught by the programme. 
There is evidence which suggests that if the programme and the syllabus had 
been identical in content, then the machine-taught students would have reached 
the same standard in a shorter time than the classroom students. 

78. A number of general conclusions can be drawn from the Melksham 
investigations. First, the majority of the students were quite happy to use a 
teaching machine for about four hours each day on a single subject for several 
weeks at a time, whereas six or eight hours each day appeared to be excessive. 
However, during the time that they were using the machines very little support 
was required of the instructor, although they prefererd to have an instructor 
available who could answer questions and get them out of difficulties. Secondly, 
since the experiment suggested that machines could teach ait least as well as 
conventional instruction, a case can be made for using programmed learning 
on economic grounds alone. Savings could be made either by increasing the 
productivity of the teacher, or by decreasing the time during which the students 
are being trained. Lastly, one particular problem that the Melksham experi- 
ment highlighted is to be found in the difficulty that was experienced in allow- 
ing students to progress at their own particular speed. It is not satisfactory to 
insert a programmed phase into the middle of a non-programmed course, since 
the resulting administrative and organisational problems are almost insoluble. 
It is also difficult to provide students with a strong enough incentive to com- 
plete the programmed phase in the minimum time of which they are capable. 

79. The work at Melksham using AutoTutors was continued in 1964. The 
findings indicate that, using a programme more similar in content to that of 
classroom instruction, the attainment under programmed instruction was 
increased. 



Fault-Finding Training 

80. An opportunity occurred in March, 1963, to explore the use of pro- 
grammed devices in an experimental fault-finding course, which was designed 
to teach a class of eleven Royal Air Force mechanics to diagnose faults to 
component level on an airborne radar equipment. Five types of programmed 
instruction were employed: workbook exercises, punchboard exercises, 
scrambled texts, ‘cheatproof’ scrambled texts, and a linear programme used in 
a home-made teaching machine. 

81. It was found that a class using the workbooks needed the full supervision 
of an instructor, who also corrected student exercises and provided remedial 
teaching. The punchboard was easy to prepare and use, and the students’ 
attitudes were generally favourable towards it. A combined use of workbook 
and punchboard left the instructor with the sole role of providing help and 
guidance. The majority of the students thought that the scrambled texts were 
useful, but they would have preferred less of them; indeed, some of the stud- 
ents preferred the punchboard to the scrambled texts. Scrambled texts with 
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‘cheatproof’ devices functioned satisfactorily, although a number of students 
complained that the device was an unnecessary complication. Generally, the 
students were not overwhelmingly impressed by the linear teaching machine, 
and they objected to the small steps in the programme. This trial demon- 
strated that programmed devices could be used in the teaching of fault-finding, 
and that simple devices can simulate some functions of complicated electronic 
equipment. Since the provision of sufficient electronic hardware for training 
purposes is expensive, the use of P.I. in this area of training has much to 
recommend it. 



The Halton Experiment 

82. In November, 1964, an experiment was mounted at Royal Air Force, 
Halton, where one hundred and fifty-eight apprentices worked their way 
through a mathetics programme on the “Principles of Automatic Car Trans- 
mission Systems”. The students were divided into five matched groups on the 
basis of a pre-test, aptitude tests, age, trade, and marks in Mathematics, Science 
and English. Four editions of the programme were prepared. Group 1 used 
the full mathetics programme, Group 4 used a special edition consisting of 
only the main teaching pages of the programme — that is to say, the pro- 
gramme used by Group 4 was one-third the size of the programme used by 
Group 1 since all practice and revisionary frames had been removed. Group 2 
used an edition which consisted of only the main teaching pages (demonstra- 
tion exercises) and prompting frames, whilst Group 3 used an edition which 
consisted of the main teaching pages and criterion or key frames (release exer- 
cises). Students worked through the programme as given, completing progress 
tests at the end of each of the four sections into which the programme was 
divided. When they had completed the programme, the students answered two 
post-tests • — the first an objective one using multiple choice questions, and the 
second a subjective one using open-ended questions. An attitude survey was 
also administered in order to determine the relevant ease or difficulty of the 
programme. 



Table 1 1 : A Summary of the Findings of the Royal Air Force 
Halton Experiment 





Mean 


Group 1 


Group 2 


Group 3 


Group 4 


Group 5 


N 




36 


36 


36 


36 


14 


Pre-test 


20-4% 


19-7% 


20-3% 


22-2% 


20-7% 


20-3% 


Post-tests 

Objective 

Subjective 


93-0% 

87-7% 




91*8% 

86-7% 


93-9% 

88-9% 




26-9% 


Retention tests 
Objective 
Subjective 
Transfer 


68-5^ 

71*5% 

47-8% 


70-9% 

73-5% 

48-6% 


691 % 
71*8% 
51-9% 


70- 1% 

71- 3^ 
44-7% 


69*2 °4 
45-9% 


27-1% 

15*3% 

4-0% 


Time to complete 
Programme 


182 

mins. 


220 

mins. 


183 

mins. 


184 

mins. 


140 

mins. 
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83. The results of the experiment are outlined in Table 11. There was no 
significant difference between the achievement obtained by any of the four 
groups, measured either by the objective or subjective post-tests. The practice 
effect of completing the same examination a number of times is indicated by 
the results obtained by Group 5, who — whilst they did not work through the 
programme — completed every examination. There was a significant differ- 
ence, however, in the time taken to complete the programme, since Group 4— 
having far less material to work through — completed the programme in a 
much shorter time. Retention tests were administered five weeks after the 
programme had been completed. The objective test was the same one that 
had been used as a pre-test and as a post-test, the subjective test was the same 
examination paper as the subjective post-test, whilst the transfer test was a new 
paper specially written for that phase of the examination. The main aim of 
the paper was not to test material that had been directly taught by the pro- 
gramme but to test the ability of the students to apply what they had learnt to 
new and novel situations. 

84. It is still too early to draw too many conclusions about the Halton study, 
and the experiment will be repeated with a less able group. There are a number 
of lessons, however, that can be drawn, particularly from the point of view of 
programme writing. Regardless of which edition the students used, the pro- 
gramme was reported to be interesting and challenging. Since the programme 
used by Group 4 was one-third the size of the programme used by Group 1, 
one can conclude that the shorter programme — regardless of the very steep 
gradient of its material — was the more efficient instrument. Furthermore, 
since the overt responses required by the students working through the shortest 
programme were in every case copy responses, it would seem that such a 
response form need not be as inefficient as has been so often suggested. 

Further Experiments 

85. Three further experiments at Royal Air Force, St. Athan, Royal Air 
Force, Newton, and Royal Air Force, Compton Bassett, must also be men- 
tioned. 

86. The Temac Algebra programme was used with Boy Entrant trainees aL 
St. Athan. The general conclusion was that the students using the programme 
obtained a result equivalent to that obtained by normal methods with very 
much less instructor supervision. This was achieved in spite of the fact that 
the Temac programme covers a more extensive syllabus than the normal 
course. However, later classes complained that the Temac programme was 
uninteresting, and asked to cease work on it. A scrambled programme has 
been accepted by the students with more enthusiasm. 

87. Two linear programmes, “Basic Electricity” and “Basic Electronics”, 
published by McGraw-Hill, were used to teach a class of post-graduate engine 
and airframe fitters at RAF Newton. The experiment showed that programmed 
books could be used successfully at a considerable saving of instructor time, 
and that the use of linear programmes in book form over a six week period 
was received with enthusiasm by the students. It thus counterbalances the 
implication from the St. Athan trial that linear programmes are unsuitable 
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lor extended periods of study. The experiment also demonstrated once again 
the difficulty of using a programme not entirely similar to the normal syllabus 
of the course. 

88. The need to programme a complete course led to the selection for this 
purpose of a one week post-graduate course on a communication equipment 
at RAF Cdmpton Bassett. A task analysis of this course was carried out as a 
first step in writing the programme. The analysis revealed, however, that such 
maintenance work as was necessary for the equipment could be learnt on the 
job, and no formal training course was required. Accordingly the programme 
was abandoned and the course disestablished. 



Programmed Instruction at the Royal Air Force School of Education 

89. Whilst all this experimentation was taking place, the Royal Air Force 
School of Education was concerning itself with the difficulties of writing and 
presenting programmed materials to students who were not inherently bookish 
or intellectual in their approach to learning. It was felt that many teaching 
machines force programmers to rely far too much on the written word, 
whereas many relationships and discriminations ought to be discovered by 
the students themselves through observing actual working instruments and 
equipments. The final result of this investigation was the invention of the 
Empirical Tutor, now available on the commercial market and used by a 
number of universities and industrial organisations. The machine, which uses 
a linear or simple branching programme and employs constructed response 
and multiple-choice response modes, automatically controls a whole range of 
training equipment. The student is thus encouraged to take a far more active 
role in the learning process by being enabled to make full use of his powers of 
observation and deliberation, evaluation and judgment. In this way, the 
teacher’s fundamental role in the teaching process is not usurped; instead, he 
is freed from many of his routine and tiresome chores so that there is increased 
time available for inspirational teaching and individual tuition. 

90. Once the problem of designing a completely flexible teaching machine 
had been tackled, the difficulty of constructing and writing programmes for it 
became even more difficult — the very flexibility of the machine created even 
more problems! Finally, a complete system of programme construction was 
developed and subsequently published as “Programmed Learning in Perspec- 
tive”. In this system, programming is conceived as a sixfold process: study of 
the syllabus, reconciliation of the syllabus with the needs of students and with 
the specific aims of training, preparation of a scheme of work, collection and 
organisation of material, construction of a matrix and flow diagram, and the 
final frame writing. 

91. In following through these stages, the subject material is fashioned and 
structured for learning by the target population for whom the programme is 
being written. Relationships between units of information are recorded on a 
matrix, which indicates an order in which the material should be presented, 
the points at which branches should diverge and converge, the position of 
revisionary sequences, and the number of frames necessary to teach each rule 
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and establish each concept. Whilst there are many ways of writing a pro- 
gramme, it has been found that this scheme helps to prevent an unnecessary 
proliferation of frames, reduces the amount of time necessary in preparing 
and writing programmes, and helps overcome some of the problems of training 
programme writers. 

Current Royal Air Force Studies 

92. The period of initial investigation concerned with evaluating programmed 
learning in the Royal Air Force is now at an end. A new phase is just begin- 
ning. This consists of an experimental application of programmed learning 
to some of the current training tasks on a restricted and discriminate basis, 
with the writing of programmes by teams and individuals organised by the 
use of a network analysis. Four projects in the use of P.I. are in progress : 

(a) The Science A & B syllabus for the RAF Education test is being pro- 
grammed for the use of airmen studying for this examination at opera- 
tional stations. 

(b) Commercial programmes in algebra and mathematics are being made 
available for airmen taking the GCE “O” level mathematics examina- 
tion. The algebra programme is in book form, whilst the mathematics 
programme is presented by a machine. 

(c) A one week post-graduate course on transistors has been prepared for 
presentation both by scrambled book and by teaching machine. The 
programme will be used on operational stations by students who would 
otherwise have attended a training school for this course. 

(d) Teams of programme writers are being set up to prepare programmes 
for technical training in four areas. When complete, these programmes 
may be used either at the student’s home station or as a replacement 
for conventional instruction at the training school. 

93. The application of programmed learning to these Royal Air Force train- 
ing situations holds the promise of improved training either by reducing 
existing training costs, or by increasing the level of students attainment, or 
by providing instruction in areas where it has been difficult to provide con- 
ventional instruction. While there are many problems of an administrative and 
organisational character to overcome, the successful use of P.I. in these situ- 
ations would result in more efficient and effective training. 
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Introductory Note 

This report was submitted in May 1965 by a Working Party set up by the 
Advisory Panel of Service Senior Psychologists to enquire into research and 
development issues relevant to Service training, with the following terms of 
reference: 

“ To examine the current extent of research into Service training, 
starting with Programmed Instruction. To report on progress and 
likely trends and to recommend to the Advisory Panel of Service 
Senior Psychologists such measures as should be taken to ensure that 
future problems are identified and investigated.” 

Because of the potential interest of the contents of the report to those con- 
cerned with problems of education and training outside the Services as well as 
within, general publication was considered worthwhile. 



Outline of the Report 

Programmed Instruction (P.I.) is often, but not necessarily, associated with 
teaching machines. It is essentially self-instruction; but is designed to allow 
more effective and economic training, rather than simply to replace human 
instruction. Since its introduction in this country only six years ago, it has 
developed into a powerful new method, applicable to a wide field of military 
education and training. 

Instruction can be said to be programmed when : 

(a) There is a clear statement of exactly what a man will be able to do at 
the end of it. 

(b) Everything which has to be learned is broken down into discrete but 
associated items, and is taught through a sequence of steps (‘frames’). 

(e) Trainees follow an actual sequence of frames which is determined for 
them according to their individual needs on the one hand, and to the rules of 
programming technique on the other. 

(d) Trainees are actively responding all the time, and there is a continual 
and immediate check on their accuracy and progress. 

(e) Subsequent steps in instruction are never taken until success has been 
demonstrated at preceding items. 

The Service departments have been pioneers in the United Kingdom with 
respect both to programme writing and to studies of productive ways of using 
P.I. Much of their work has been described in official reports and journal 
articles. However, in order to give a concise account of this research and 
development work, Appendix 1 of the present report summarises the experi- 
ments and programme- writing activity carried out by each Service between 1961 
and 1965. 
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So far most applications of P.X. have concentrated on scholastic subjects. 
However, its principles are not confined to this area. They are equally valid for 
manual and perceptual tasks. Current service applications in fact cover a wide 
spectrum, from the technical training of electronic technicians to courses on 
managerial skill. 

Early in the development of P.I., it was realised that efficiency of training 
is a function of many factors; the quality of instruction is only one of these. 
Practical applications of P.I. have given rise to the wider concept of a ‘tech- 
nology of training’. The nature of this technology is discussed in the report. It 
includes the systematic derivation of training objectives from the study of 
operational task requirements; the assessment of knowledge and ability among 
potential students; specification of contents, methods, and materials, of training 
courses in the light of the foregoing; and, finally, the assessment of training 
efficiency in terms of performance standards and cost-effectiveness. 

Future research and development trends in P.I. are discussed in some 
detail. Among developments already under way (and, indeed, well advanced in 
the United States), the Working Party points to computer-assisted instruction 
as likely to make an impact in due course, e.g. in providing instruction which 
is adapted to individual differences, in assisting human instructors to ‘pro- 
gramme’ their courses, and in helping to identify the best instructional 
‘strategies’ for particular training needs. 

A widespread Service use of P.I. offers potential savings in training costs 
due to factors such as: 

(a) reduced demands on highly qualified instructors, 

(b) reduction of training time, due to increased rates of learning, 

(c) attainment of higher standards of knowledge and proficiency, 

(d) increased ‘through-put’ of trainees. 

While all these gains have yet to be achieved in a single project, the growing 
use of P.I. as a routine training method increases the likelihood of their com- 
plete realisation. 



J. C. Penton, Chairman 
J. B. Parry 
N. A. B. Wilson 

Advisory Panel of Service Senior Psychologists, Ministry of Defence, London 
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